Error when trying to quantize GGUF.

[Samdoses]

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pandas-stubs 2.0.3.230814 requires numpy>=1.25.0; python_version >= "3.9", but you have numpy 1.24.4 which is incompatible.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.25.3 which is incompatible.
torchaudio 2.2.1+cu121 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible.
torchtext 0.17.1 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible.
torchvision 0.17.1+cu121 requires torch==2.2.1, but you have torch 2.1.2 which is incompatible.
Successfully installed einops-0.7.0 gguf-0.6.0 numpy-1.24.4 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.18.1 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.1.105 protobuf-4.25.3 torch-2.1.2 triton-2.1.0
WARNING: The following packages were previously imported in this runtime:
[numpy]
You must restart the runtime in order to use newly installed versions.

[Samdoses]

I understand that the wrong version of several dependencies are being installed from the requirements.txt of llama.cpp. Is there any way to manually set the misbehaving dependencie's version myself in the collab code?

Thanks!

[mlabonne]

Yes, you can add a line after the installation of the requirements: !pip install your-library

…

On Sat, Apr 20, 2024, 17:39 Samdoses ***@***.***> wrote: I understand that the wrong version of several dependancies are being installed from the requirements.txt of llama.cpp. Is there any way to manually set the dependancy version myself in the collab code? — Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATL5EGS4AOHWEOI37WJH6BLY6KK33AVCNFSM6AAAAABGQTW3QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXG4ZDIOBQG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Samdoses]

I tried installing the dependencies, but they seem to be uninstalled immediately. Is there a way to prevent this and force the code to use the updated versions instead of those specified in the requirements file?

Attempting uninstall: protobuf
Found existing installation: protobuf 3.20.3
Uninstalling protobuf-3.20.3:
Successfully uninstalled protobuf-3.20.3
Attempting uninstall: numpy
Found existing installation: numpy 1.25.2
Uninstalling numpy-1.25.2:
Successfully uninstalled numpy-1.25.2
Attempting uninstall: torch
Found existing installation: torch 2.2.1+cu121
Uninstalling torch-2.2.1+cu121:
Successfully uninstalled torch-2.2.1+cu121

[jazelly]

Hi, have you considered using venv or conda? Looks like you got some existing deps that are unhappy with the requirements.txt that you are trying to install