[Bug Report] All fine tunes for Mistral 7b using sagemaker jumpstart are currently failing.

[aadupirn]

All fine tunes for Mistral 7b using sagemaker jumpstart are currently failing with:

"ImportError: cannot import name 'insecure_hashlib' from 'huggingface_hub.utils' (/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/init.py)"

After changing nothing on my end fine tuning started to fail after many previous successful fine tunes.

To reproduce
Try to fine tune any model using the guides here:
https://aws.amazon.com/blogs/machine-learning/fine-tune-and-deploy-mistral-7b-with-amazon-sagemaker-jumpstart/

[aadupirn]

I was able to work around this:

The bug is still in sagemaker but I was able to workaround it but downloading and unpacking the source from AWS that is used and manually fixing deps by building new linux whl's that don't break. Using the latest whls for linux 2014 for transformers, tokenizer and huggingface_hub will fix it.

[tuneshverma]

I am also facing the same issue, can you please elaborate more, how you fixed the issue. I am fairly new to this.
Did you get the sagemaker image and create a container and then did the required change and created new image from that and used that, if yes then from where did you get the image and how to use the new image with sagemaker train pipeline.
Thanks in advance.

[tuneshverma]

solved the issue by changing the version of transformers to latest which is 4.37.2, which updated the tokenizer and huggingface_hub