Skip to content

Issues: NVIDIA/TensorRT-LLM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

549 Open 760 Closed
549 Open 760 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How to test the benchmark of Llama3 and Vicuna2 of TensorRT-LLM by benchmark.py
#1597 opened May 14, 2024 by Ourspolaire1
Fail to run Mixtral 8x7b with tp size 4 on w4a16 bug Something isn't working
#1596 opened May 14, 2024 by gloritygithub11
2 of 4 tasks
rotary_scaling build command doesnt work bug Something isn't working
#1595 opened May 13, 2024 by avianion
4 tasks done
enableBlockReuse option is not available for tensorrt_llm.runtime.ModelRunner bug Something isn't working
#1594 opened May 13, 2024 by yupbank
2 of 4 tasks
1
Question: Do we support input log probabilties with C++ inflight backend?
#1593 opened May 13, 2024 by sindhuvahinis
Classification with LoRA and LLAMA/Mistral example bug Something isn't working
#1592 opened May 13, 2024 by bjayakumar
2 of 4 tasks
Feature Request: "Model Zoo" for quantization
#1591 opened May 13, 2024 by atyshka
Top-P sampling occasionally produces invalid tokens bug Something isn't working
#1590 opened May 13, 2024 by AlessioNetti
4 tasks done
convert qwen 110b gptq checkpoint的时候，qkv_bias 的shape不能被3整除
#1589 opened May 13, 2024 by CallmeZhangChenchen
Inference Qwen1.5-14B with 2x RTX4090D failed based main branch
#1588 opened May 13, 2024 by Fred-cell
TllmXqaJit runtime error when build Yi-6B fp8 with TRTLLM-0.10.0.dev2024050700 bug Something isn't working
#1586 opened May 13, 2024 by kimbaol
2 of 4 tasks
3
[Quantization] [mixtral_8x22B] NotImplementedError: Cannot copy out of meta tensor; no data! bug Something isn't working
#1585 opened May 13, 2024 by Godlovecui
2 of 4 tasks
2
getPluginCreator could not find plugin: Gemmtensorrt_llm version: 1 bug Something isn't working
#1584 opened May 13, 2024 by gloritygithub11
2 of 4 tasks
prepare_dataset.py issue
#1582 opened May 12, 2024 by Fred-cell
How to build int4_gptq on Mixtral 8x7b
#1581 opened May 12, 2024 by gloritygithub11
Fail to build int4_awq on Mixtral 8x7b bug Something isn't working
#1580 opened May 12, 2024 by gloritygithub11
2 of 4 tasks
Is there a version of nvidia-ammo for Windows?
#1579 opened May 11, 2024 by Milesld
Failed to quantize Starcoder2 with FP8 bug Something isn't working
#1578 opened May 11, 2024 by wxsms
2 of 4 tasks
How to set the initial kv cache length?
#1577 opened May 11, 2024 by liminn
Error occured when running medusa inference.
#1575 opened May 10, 2024 by littletomatodonkey
Does enc-dec model support inflight bathing? question Further information is requested
#1573 opened May 10, 2024 by Oldpan
@symphonylyh
Why is the calculation result of tensorrt-llm version llava1.5 different from the output of HF? triaged Issue has been triaged by maintainers
#1572 opened May 10, 2024 by bleedingfight
2 of 4 tasks
@byshiue
2
Qwen-7B build failed on Windows with trtllm-0.9.0 bug Something isn't working
#1571 opened May 10, 2024 by bigbigQI
4 tasks
Why H200 shows only few improvement over H100 on Mistral-7B? perf Issue about performance number
#1570 opened May 10, 2024 by shixuan94
@kaiyux
Missing logits in Executor API when using return_generation_logits bug Something isn't working triaged Issue has been triaged by maintainers
#1569 opened May 10, 2024 by AlessioNetti
2 of 4 tasks
@trevor-m
ProTip! Find all open issues with in progress development work with linked:pr.