We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I'm evaluating with the officially supported tasks/models/datasets.
python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))" {'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda', 'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0', 'GPU 0,1,2,3': 'Tesla P40', 'MMEngine': '0.10.4', 'MUSA available': False, 'NVCC': 'Cuda compilation tools, release 11.7, V11.7.64', 'OpenCV': '4.9.0', 'PyTorch': '2.0.1+cu117', 'PyTorch compiling details': 'PyTorch built with:\n' ' - GCC 9.3\n' ' - C++ Version: 201703\n' ' - Intel(R) oneAPI Math Kernel Library Version ' '2023.1-Product Build 20230303 for Intel(R) 64 ' 'architecture applications\n' ' - Intel(R) MKL-DNN v2.7.3 (Git Hash ' '6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n' ' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n' ' - LAPACK is enabled (usually provided by ' 'MKL)\n' ' - NNPACK is enabled\n' ' - CPU capability usage: AVX2\n' ' - CUDA Runtime 11.7\n' ' - NVCC architecture flags: ' '-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n' ' - CuDNN 8.5\n' ' - Magma 2.6.1\n' ' - Build settings: BLAS_INFO=mkl, ' 'BUILD_TYPE=Release, CUDA_VERSION=11.7, ' 'CUDNN_VERSION=8.5.0, ' 'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, ' 'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 ' '-fabi-version=11 -Wno-deprecated ' '-fvisibility-inlines-hidden -DUSE_PTHREADPOOL ' '-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER ' '-DUSE_FBGEMM -DUSE_QNNPACK ' '-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK ' '-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC ' '-Wall -Wextra -Werror=return-type ' '-Werror=non-virtual-dtor -Werror=bool-operation ' '-Wnarrowing -Wno-missing-field-initializers ' '-Wno-type-limits -Wno-array-bounds ' '-Wno-unknown-pragmas -Wunused-local-typedefs ' '-Wno-unused-parameter -Wno-unused-function ' '-Wno-unused-result -Wno-strict-overflow ' '-Wno-strict-aliasing ' '-Wno-error=deprecated-declarations ' '-Wno-stringop-overflow -Wno-psabi ' '-Wno-error=pedantic -Wno-error=redundant-decls ' '-Wno-error=old-style-cast ' '-fdiagnostics-color=always -faligned-new ' '-Wno-unused-but-set-variable ' '-Wno-maybe-uninitialized -fno-math-errno ' '-fno-trapping-math -Werror=format ' '-Werror=cast-function-type ' '-Wno-stringop-overflow, LAPACK_INFO=mkl, ' 'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, ' 'PERF_WITH_AVX512=1, ' 'TORCH_DISABLE_GPU_ASSERTS=ON, ' 'TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, ' 'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, ' 'USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, ' 'USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, ' 'USE_OPENMP=ON, USE_ROCM=OFF, \n', 'Python': '3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0]', 'TorchVision': '0.15.2+cu117', 'numpy_random_seed': 2147483648, 'opencompass': '0.2.4+19d7e63', 'sys.platform': 'linux'}
python run.py --datasets cmb_gen --hf-path /opt/models/Qwen1.5-0.5B-Chat --model-kwargs device_map='auto' --tokenizer-kwargs padding_side='left' truncation='left' use_fast=False --max-out-len 100 --max-seq-len 2048 --batch-size 8 --no-batch-padding --num-gpus 0 05/11 10:27:04 - OpenCompass - INFO - Loading cmb_gen: configs/datasets/cmb/cmb_gen.py 05/11 10:27:04 - OpenCompass - INFO - Loading example: configs/summarizers/example.py 05/11 10:27:04 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored. 05/11 10:27:04 - OpenCompass - INFO - Partitioned into 7 tasks. launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5] on CPU launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb] on CPU 0%| | 0/7 [00:00<?, ?it/s] 05/11 10:27:31 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2.out 14%|██████████████████ | 1/7 [00:27<02:45, 27.63s/it] 05/11 10:27:34 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb.out 29%|████████████████████████████████████ | 2/7 [00:30<01:04, 12.91s/it] 05/11 10:27:35 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5.out 43%|██████████████████████████████████████████████████████ | 3/7 [00:31<00:30, 7.71s/it] 05/11 10:27:36 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3.out 57%|████████████████████████████████████████████████████████████████████████ | 4/7 [00:32<00:14, 4.93s/it] 05/11 10:27:36 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4.out 05/11 10:27:37 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1.out 86%|████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 6/7 [00:33<00:02, 2.57s/it] 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0] fail, see ./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0.out 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:33<00:00, 4.84s/it] 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_0] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_1] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_2] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_3] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_4] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_5] failed with code 1 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb] failed with code 1 05/11 10:27:38 - OpenCompass - INFO - Partitioned into 2 tasks. launch OpenICLEval[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb] on CPU launch OpenICLEval[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test] on CPU 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:11<00:00, 5.69s/it] dataset version metric mode opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat
cmb - - - - cmb_test - - - - 05/11 10:27:49 - OpenCompass - INFO - write summary to /software/opencompass/outputs/default/20240511_102704/summary/summary_20240511_102704.txt 05/11 10:27:49 - OpenCompass - INFO - write csv to /software/opencompass/outputs/default/20240511_102704/summary/summary_20240511_102704.csv (opencompass) root@jkha-W580-G20:/software/opencompass# echo $? 0
python run.py --datasets cmb_gen_dfb5c4 --hf-path /opt/models/Qwen1.5-0.5B-Chat --model-kwargs device_map='auto' --tokenizer-kwargs padding_side='left' truncation='left' use_fast=False --max-out-len 100 --max-seq-len 2048 --batch-size 8 --no-batch-padding --num-gpus 0
05/11 10:37:47 - OpenCompass - INFO - write summary to /software/opencompass/outputs/default/20240511_103700/summary/summary_20240511_103700.txt 05/11 10:37:47 - OpenCompass - INFO - write csv to /software/opencompass/outputs/default/20240511_103700/summary/summary_20240511_103700.csv
cmb_gen cmb_gen_dfb5c4
The text was updated successfully, but these errors were encountered:
tonysy
No branches or pull requests
Prerequisite
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))"
{'CUDA available': True,
'CUDA_HOME': '/usr/local/cuda',
'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',
'GPU 0,1,2,3': 'Tesla P40',
'MMEngine': '0.10.4',
'MUSA available': False,
'NVCC': 'Cuda compilation tools, release 11.7, V11.7.64',
'OpenCV': '4.9.0',
'PyTorch': '2.0.1+cu117',
'PyTorch compiling details': 'PyTorch built with:\n'
' - GCC 9.3\n'
' - C++ Version: 201703\n'
' - Intel(R) oneAPI Math Kernel Library Version '
'2023.1-Product Build 20230303 for Intel(R) 64 '
'architecture applications\n'
' - Intel(R) MKL-DNN v2.7.3 (Git Hash '
'6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)\n'
' - OpenMP 201511 (a.k.a. OpenMP 4.5)\n'
' - LAPACK is enabled (usually provided by '
'MKL)\n'
' - NNPACK is enabled\n'
' - CPU capability usage: AVX2\n'
' - CUDA Runtime 11.7\n'
' - NVCC architecture flags: '
'-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n'
' - CuDNN 8.5\n'
' - Magma 2.6.1\n'
' - Build settings: BLAS_INFO=mkl, '
'BUILD_TYPE=Release, CUDA_VERSION=11.7, '
'CUDNN_VERSION=8.5.0, '
'CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, '
'CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 '
'-fabi-version=11 -Wno-deprecated '
'-fvisibility-inlines-hidden -DUSE_PTHREADPOOL '
'-DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER '
'-DUSE_FBGEMM -DUSE_QNNPACK '
'-DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK '
'-DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC '
'-Wall -Wextra -Werror=return-type '
'-Werror=non-virtual-dtor -Werror=bool-operation '
'-Wnarrowing -Wno-missing-field-initializers '
'-Wno-type-limits -Wno-array-bounds '
'-Wno-unknown-pragmas -Wunused-local-typedefs '
'-Wno-unused-parameter -Wno-unused-function '
'-Wno-unused-result -Wno-strict-overflow '
'-Wno-strict-aliasing '
'-Wno-error=deprecated-declarations '
'-Wno-stringop-overflow -Wno-psabi '
'-Wno-error=pedantic -Wno-error=redundant-decls '
'-Wno-error=old-style-cast '
'-fdiagnostics-color=always -faligned-new '
'-Wno-unused-but-set-variable '
'-Wno-maybe-uninitialized -fno-math-errno '
'-fno-trapping-math -Werror=format '
'-Werror=cast-function-type '
'-Wno-stringop-overflow, LAPACK_INFO=mkl, '
'PERF_WITH_AVX=1, PERF_WITH_AVX2=1, '
'PERF_WITH_AVX512=1, '
'TORCH_DISABLE_GPU_ASSERTS=ON, '
'TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, '
'USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, '
'USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, '
'USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, '
'USE_OPENMP=ON, USE_ROCM=OFF, \n',
'Python': '3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0]',
'TorchVision': '0.15.2+cu117',
'numpy_random_seed': 2147483648,
'opencompass': '0.2.4+19d7e63',
'sys.platform': 'linux'}
Reproduces the problem - code/configuration sample
python run.py --datasets cmb_gen
--hf-path /opt/models/Qwen1.5-0.5B-Chat
--model-kwargs device_map='auto'
--tokenizer-kwargs padding_side='left' truncation='left' use_fast=False
--max-out-len 100
--max-seq-len 2048
--batch-size 8
--no-batch-padding
--num-gpus 0
05/11 10:27:04 - OpenCompass - INFO - Loading cmb_gen: configs/datasets/cmb/cmb_gen.py
05/11 10:27:04 - OpenCompass - INFO - Loading example: configs/summarizers/example.py
05/11 10:27:04 - OpenCompass - WARNING - SlurmRunner is not used, so the partition argument is ignored.
05/11 10:27:04 - OpenCompass - INFO - Partitioned into 7 tasks.
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5] on CPU
launch OpenICLInfer[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb] on CPU
0%| | 0/7 [00:00<?, ?it/s] 05/11 10:27:31 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_2.out
14%|██████████████████ | 1/7 [00:27<02:45, 27.63s/it] 05/11 10:27:34 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb.out
29%|████████████████████████████████████ | 2/7 [00:30<01:04, 12.91s/it] 05/11 10:27:35 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_5.out
43%|██████████████████████████████████████████████████████ | 3/7 [00:31<00:30, 7.71s/it] 05/11 10:27:36 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_3.out
57%|████████████████████████████████████████████████████████████████████████ | 4/7 [00:32<00:14, 4.93s/it] 05/11 10:27:36 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_4.out
05/11 10:27:37 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_1.out
86%|████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 6/7 [00:33<00:02, 2.57s/it] 05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/local.py - _launch - 206 - task OpenICLInfer[opencompass.models.huggingface.Hugg ingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0] fail, see
./outputs/default/20240511_102704/logs/infer/opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test_0.out
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:33<00:00, 4.84s/it]
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_0] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_1] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_2] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_3] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_4] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb_test_5] failed with code 1
05/11 10:27:38 - OpenCompass - ERROR - /software/opencompass/opencompass/runners/base.py - summarize - 64 - OpenICLInfer[opencompass.models.huggingface.HuggingFa ce_models_Qwen1.5-0.5B-Chat/cmb] failed with code 1
05/11 10:27:38 - OpenCompass - INFO - Partitioned into 2 tasks.
launch OpenICLEval[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb] on CPU
launch OpenICLEval[opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat/cmb_test] on CPU
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:11<00:00, 5.69s/it]
dataset version metric mode opencompass.models.huggingface.HuggingFace_models_Qwen1.5-0.5B-Chat
cmb - - - -
cmb_test - - - -
05/11 10:27:49 - OpenCompass - INFO - write summary to /software/opencompass/outputs/default/20240511_102704/summary/summary_20240511_102704.txt
05/11 10:27:49 - OpenCompass - INFO - write csv to /software/opencompass/outputs/default/20240511_102704/summary/summary_20240511_102704.csv
(opencompass) root@jkha-W580-G20:/software/opencompass# echo $?
0
Reproduces the problem - command or script
python run.py --datasets cmb_gen_dfb5c4
--hf-path /opt/models/Qwen1.5-0.5B-Chat
--model-kwargs device_map='auto'
--tokenizer-kwargs padding_side='left' truncation='left' use_fast=False
--max-out-len 100
--max-seq-len 2048
--batch-size 8
--no-batch-padding
--num-gpus 0
Reproduces the problem - error message
05/11 10:37:47 - OpenCompass - INFO - write summary to /software/opencompass/outputs/default/20240511_103700/summary/summary_20240511_103700.txt
05/11 10:37:47 - OpenCompass - INFO - write csv to /software/opencompass/outputs/default/20240511_103700/summary/summary_20240511_103700.csv
Other information
cmb_gen cmb_gen_dfb5c4
The text was updated successfully, but these errors were encountered: