add mx quant #1728
add mx quant #1728
Conversation
⛈️ Required checks status: Has failure 🔴
Groups summary🟢 Code Scan Tests workflow
These checks are required after the changes to 🟢 Model Tests 3x workflow
These checks are required after the changes to 🟢 Unit Tests 3x-TensorFlow workflow
These checks are required after the changes to 🔴 Unit Tests 3x-PyTorch workflow
These checks are required after the changes to 🔴 Unit Tests 3x-ONNXRT workflow
These checks are required after the changes to Thank you for your contribution! 💜
|
...x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx/run_clm_no_trainer.py
Show resolved
Hide resolved
mengniwang95
force-pushed
the
mengni/torch_mx
branch
2 times, most recently
from
4235c72
to
988e9a8
Compare
|
@mengniwang95 any update? |
|
Does this feature target v2.6? |
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
mengniwang95
force-pushed
the
mengni/torch_mx
branch
from
0dc1788
to
867acf8
Compare
for more information, see https://pre-commit.ci
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
There was a problem hiding this comment.
Not able to run the example, here is what i did in a fresh venv
git clone https://github.com/intel/neural-compressor.git
cd neural-compressor
gh pr checkout 1728
pip install -r requirements.txt
python setup.py install
cd /neural-compressor/examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx
pip install -r requirements.txt
python3 -u run_clm_no_trainer.py --model bigscience/bloom-560m --quantize --accuracy --tasks lambada_openai --w_dtype fp4 --woq
then i get the error
2024-05-09 13:28:52 [INFO][algorithm_entry.py:512] Quantize model with the mx quant algorithm.
2024-05-09 13:29:01 [INFO][run_clm_no_trainer.py:61] Quantization end.
Traceback (most recent call last):
File "/home/anthony/venv_pr1728/neural-compressor/examples/3.x_api/pytorch/nlp/huggingface_models/language-modeling/quantization/mx/run_clm_no_trainer.py", line 66, in
from intel_extension_for_transformers.transformers.llm.evaluation.lm_eval import evaluate
File "/home/anthony/venv_pr1728/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/llm/evaluation/lm_eval/init.py", line 17, in
from .accuracy import cli_evaluate as evaluate
File "/home/anthony/venv_pr1728/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/llm/evaluation/lm_eval/accuracy.py", line 42, in
from intel_extension_for_transformers.transformers.llm.evaluation.lm_eval import evaluator
File "/home/anthony/venv_pr1728/lib/python3.10/site-packages/intel_extension_for_transformers/transformers/llm/evaluation/lm_eval/evaluator.py", line 29, in
import lm_eval.api.metrics
ModuleNotFoundError: No module named 'lm_eval.api'
There was a problem hiding this comment.
Thank you for your reminder, we will update the requirements.txt. Please install intel-extension-for-transformers >= 1.4.1 and lm-eval==0.4.2
for more information, see https://pre-commit.ci
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
for more information, see https://pre-commit.ci
Signed-off-by: Mengni Wang <mengni.wang@intel.com>
Type of Change
feature
Description
support mx quant
Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
UT
Dependency Change?
any library dependency introduced or removed