Skip to content

Issues: triton-inference-server/tensorrtllm_backend

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

190 Open 167 Closed
190 Open 167 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

There is a problem with llama 7B model pre-processing after using triton server bug Something isn't working
#445 opened May 8, 2024 by Graham1025
2 of 4 tasks
[BUG] coredump when process exit triggered after TRITONSERVER_ServerDelete
#443 opened May 7, 2024 by hzlushiliang
InFlightBatching seems not working bug Something isn't working
#442 opened May 6, 2024 by larme
2 of 4 tasks
Deployement failed for BERT
#440 opened May 3, 2024 by vivekjoshi556
Deploying Mixtral-8x7B-v0.1 with Triton 24.02 on A100 (160GB) raises "Cuda Runtime (out of memory)" exception bug Something isn't working
#438 opened Apr 29, 2024 by kelkarn
2 of 4 tasks
1
GptManager’s scalability issues with input & output parameters
#437 opened Apr 28, 2024 by service-kit
1
How to post sample parameters (like top_k, temperature) for triton http server bug Something isn't working
#436 opened Apr 26, 2024 by wanzhenchn
2 of 4 tasks
Encountered an error in forward function: std::bad_cast bug Something isn't working
#435 opened Apr 26, 2024 by wangqy1216
1 of 4 tasks
@schetlur-nv
1
LLama 7B model can't get longer ouput text after using triton server bug Something isn't working
#434 opened Apr 26, 2024 by XiaobingSuper
2 of 4 tasks
@schetlur-nv
max_batch_size seems to have no impact on model performance bug Something isn't working triaged Issue has been triaged by maintainers
#429 opened Apr 23, 2024 by VitalyPetrov
3 of 4 tasks
@byshiue
6
Performance Issue with return_context_logits Enabled in TensorRT-LLM bug Something isn't working
#428 opened Apr 23, 2024 by gywlssww
2 of 4 tasks
Seg fault after loaded models in official example bug Something isn't working
#425 opened Apr 20, 2024 by LeatherDeerAU
2 of 4 tasks
2
Can't launch triton server following docs, expecting [TensorRT] library version 9.2.0.5 got 9.3.0.1 bug Something isn't working
#424 opened Apr 20, 2024 by conway-abacus
2 of 4 tasks
@byshiue
5
How is GptManager used in Triton backend?
#421 opened Apr 19, 2024 by ekagra-ranjan
@byshiue
1
Performance Issue with return_context_logits Enabled in TensorRT-LLM bug Something isn't working triaged Issue has been triaged by maintainers
#419 opened Apr 19, 2024 by metterian
2 of 4 tasks
@yweng0828
1
Filtering beam_search output tensors results in a string output vs list triaged Issue has been triaged by maintainers
#418 opened Apr 18, 2024 by nikhilshandilya
@byshiue
1
Warmup Example of loading LoRa weights triaged Issue has been triaged by maintainers
#417 opened Apr 18, 2024 by TheCodeWrangler
@byshiue
2
the result use inflight_batcher_llm_client to send multiple lora weights is not same as use tensorrtllm triaged Issue has been triaged by maintainers
#413 opened Apr 17, 2024 by stifles
@byshiue
2
Feature Request: Set maximum number of in flight feature request New feature or request
#412 opened Apr 17, 2024 by TheCodeWrangler
@ncomly-nvidia
Block reuse is currently not supported with beam width > 1 triaged Issue has been triaged by maintainers
#411 opened Apr 16, 2024 by tonylek
@byshiue
3
dynamic batching not working properly with tensorrtllm_backend
#422 opened Apr 14, 2024 by gavinzb
3
Supporting beam search in streaming mode feature request New feature or request
#408 opened Apr 13, 2024 by tonylek
@ncomly-nvidia
lora_task_id, lora_weights, lora_config not found in all_models/inflight_batcher_llm/tensorrt_llm_bls/1/lib/decode.py bug Something isn't working
#406 opened Apr 12, 2024 by liao217
1 of 4 tasks
Support bfloat16 LoRa Adaptors bug Something isn't working triaged Issue has been triaged by maintainers
#403 opened Apr 11, 2024 by TheCodeWrangler
@byshiue
5
Example of LoRa weights triaged Issue has been triaged by maintainers
#399 opened Apr 9, 2024 by TheCodeWrangler
@byshiue
2
ProTip! Mix and match filters to narrow down what you’re looking for.