Skip to content

Issues: ModelTC/lightllm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

50 Open 110 Closed
50 Open 110 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[BUG] Slow Tokenizer Message is printing when the Fast Tokenizer may be in use bug Something isn't working
#407 opened May 7, 2024 by david-vectorflow
1
[Question] How does lightllm implement nopad batching?
#405 opened Apr 25, 2024 by Tomorrowdawn
请问是否有计划支持MiniCPM-V-2
#404 opened Apr 23, 2024 by xiabo0816
1
[BUG] There already is a lightllm in pypi bug Something isn't working
#380 opened Mar 26, 2024 by rlippmann
1 task
weight only int4 is slower than cutlass int4
#362 opened Mar 19, 2024 by zhoutianzi666
1
Are there any efficient way to command kill the lightllm process?
#343 opened Mar 4, 2024 by yy9996
3
Qwen-14B-INT8 face the issue: 'QwenTransformerLayerWeight' object has no attribute 'q_weight_' bug Something isn't working
#333 opened Feb 20, 2024 by wangr0031
1
[BUG] stop_words bug Something isn't working
#326 opened Feb 2, 2024 by baisechundu
[BUG] Support for DeepSeek? bug Something isn't working
#325 opened Feb 2, 2024 by suhjohn
1
是否能支持sqlcoder系列模型
#310 opened Jan 22, 2024 by 2496289471
2
Inconsistent Output between LightLLM and Transformers Inference Library bug Something isn't working
#309 opened Jan 19, 2024 by Lvjinhong
2
请问lightllm可以离线推理吗，有没有参考代码 bug Something isn't working
#308 opened Jan 19, 2024 by monkeyZhy
1 task
1
请问现在支持Yi-34B的awq 4bit部署吗？
#291 opened Jan 9, 2024 by xyfZzz
5
What is the plan to support beam search
#286 opened Jan 8, 2024 by feifeibear
4
[Feature]请帮忙提供load_from_weight_dict(weight_dict)接口。
#277 opened Jan 4, 2024 by bingo787
10
Is there any comparison of the effects related to token attention? For example, compare with page attention
#268 opened Dec 27, 2023 by skykiseki
2
[BUG] Qwen-7B-Chat AttributeError: 'LlamaSplitFuseInferStateInfo' object has no attribute 'logn_values' bug Something isn't working
#239 opened Dec 2, 2023 by exceedzhang
1 task
4
no attribute 'qkv_weight_' AttributeError when load Qwen-14B-Chat-Int4 bug Something isn't working
#234 opened Dec 1, 2023 by jarviszeng-zjc
8
[BUG] Assertation error self.config["num_attention_heads"] % self.world_size_ == 0 when not perfectly divisible bug Something isn't working
#233 opened Nov 30, 2023 by getorca
1
An error occurred while deploying the 4bit version of Yi-34B-Chat
#230 opened Nov 29, 2023 by wx971025
1
[BUG] triton 2.0.0.dev20221202 has a memory leak bug, and fix way. bug Something isn't working
#209 opened Nov 13, 2023 by hiworldwzj
高并发输出不稳定 bug Something isn't working enhancement New feature or request
#203 opened Nov 10, 2023 by GavinZhao19
4
源码复现过程中出现很多问题 bug Something isn't working
#187 opened Nov 1, 2023 by lxnlxnlxnlxnlxn
4
out of memory bug Something isn't working
#171 opened Oct 20, 2023 by enhaofrank
3
Quantization support
#163 opened Oct 16, 2023 by generalsvr
8
ProTip! What’s not been updated in a month: updated:<2024-04-13.