Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

107 Open 269 Closed
107 Open 269 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] ValueError: not enough values to unpack
#1235 opened Jun 11, 2024 by rangehow
2 tasks done
@bittersweet1999
[Bug] 不支持python3.10以上安装
#1234 opened Jun 10, 2024 by rangehow
2 tasks done
@Leymore
[Bug] Passing trust_remote_code=True will be mandatory to load this dataset from the next major release of datasets.
#1233 opened Jun 9, 2024 by chairmanQi
2 tasks done
@tonysy
[Bug] When testing on gen datasets, even if the output is empty or incorrect, unexpected scores can be obtained
#1232 opened Jun 7, 2024 by chairmanQi
2 tasks done
@tonysy
opencompass公开榜单更新[Feature]
#1231 opened Jun 7, 2024 by cobraheleah
1 task
@tonysy
1
大海捞针数据集初始化报错（ Failed to get opencompass.datasets.needlebench.origin.NeedleBenchOriginDataset）
#1229 opened Jun 6, 2024 by macheng6
2 tasks done
@kennymckormick
[Bug] run pytorch Qwen-7B-Chat with ARC-c ppl under CPU ,and result is not good
#1226 opened Jun 5, 2024 by FlexLaughing
2 tasks done
@liushz
1
[Bug] which version of the dataset should be selected When evaluating the Llama3 model,
#1223 opened Jun 3, 2024 by bullw
2 tasks done
@Leymore
[Bug] llm-compression task faild at eval stage with latest version
#1222 opened Jun 1, 2024 by mqy004
2 tasks done
@tonysy
3
[Bug] configs/datasets/agieval/agieval_mixed_713d14.py not found
#1220 opened May 31, 2024 by Galaxy-Husky
2 tasks done
@tonysy
[Bug] hf_chatglm3_6b评测AFQMC数据集时，自测结果与官方不一致。且自测结果不稳定。
#1202 opened May 28, 2024 by jieheroli
2 tasks done
@bittersweet1999
1
[Feature] config的bug，提示下载configs，然后下载了又出现以下bug
#1197 opened May 28, 2024 by 1518630367
1 task
@bittersweet1999
[Feature] 如何在needlebench 中使用api model？
#1196 opened May 28, 2024 by 1518630367
1 task
@DseidLi @bittersweet1999
1
[Feature] 支持openai/GPT4-o的评测seting
#1187 opened May 23, 2024 by Ezra-Yu
1 task
@tonysy
1
[Bug] 使用api测评时mode参数不起作用，超出max_seq_len并没有按mode切分输入
#1181 opened May 22, 2024 by wz0424
2 tasks done
@liushz
[Feature] Leval数据集少了两个config：codeU和sci_fi
#1179 opened May 21, 2024 by liuxize-dhu
1 task
@bittersweet1999
[Bug] CLUEWSC的测试结果全是 50%
#1174 opened May 21, 2024 by SefaZeng
2 tasks done
@bittersweet1999
[Feature] Add -f for evaluation stage
#1172 opened May 17, 2024 by liushz
1 task
@tonysy
[Bug] 根据官方的Quick Start运行，输出结果为空
#1168 opened May 16, 2024 by MansonHua
2 tasks done
@kennymckormick
10
[Bug] 评测 Hellaswag 失败
#1166 opened May 16, 2024 by SefaZeng
2 tasks done
2
自定义的模型API都要配置步骤有哪些，除了写config文件外
#1165 opened May 15, 2024 by zhe123tc
1 task
@liushz
3
[Bug] RACE数据集在hf-llama-7b上的评测结果异常
#1164 opened May 15, 2024 by Li-jiaxian
2 tasks done
@liushz
[Feature] MMLU-PRO
#1161 opened May 15, 2024 by Ezra-Yu
1 task
@liushz
1
the prompt for triviaqa dataset
#1158 opened May 15, 2024 by mengrusun
1 task done
@liushz
[Bug] run with transformers==4.40.2, error "HuggingFacewithChatTemplate does not support ppl-based evaluation".
#1157 opened May 15, 2024 by zhulinJulia24
2 tasks done
@Leymore
2
ProTip! Updated in the last three days: updated:>2024-06-08.