Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add instructions about setting TGI(S) parameters #74

Open
bdattoma opened this issue Sep 5, 2023 · 2 comments
Open

Add instructions about setting TGI(S) parameters #74

bdattoma opened this issue Sep 5, 2023 · 2 comments
Assignees
@dtrifiro
Labels
kind/documentation Improvements or additions to documentation

Comments

@bdattoma
Copy link
Contributor

bdattoma commented Sep 5, 2023 •
edited

Users may need to set particular TGI(S) parameter when using Caikit+TGIS runtime on KServe. An example is the model timeout parameter which can be necessary to be tweaked based on the model size.

We should document the procedure in our docs

Additionally, in a future UI effort, this option should be present in the user interface
@bdattoma
Copy link
Contributor Author

bdattoma commented Sep 5, 2023

@MelissaFlinn @Jooho

@heyselbi heyselbi added the kind/documentation Improvements or additions to documentation label Sep 6, 2023
@dtrifiro
Copy link
Contributor

Related #115

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dtrifiro dtrifiro

Labels
kind/documentation Improvements or additions to documentation
Projects
Internal tracking
Status: No status
ODH Feature Tracking
Status: No status
ODH Model Serving Planning
Status: To-do/Groomed
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dtrifiro @heyselbi @bdattoma