Unable to install Requirements in google colab

[pintersebastian]

Hi!

I have been stuck here for some while: i am unable to install all the requierements from the provided file. I removed torch as it says torch comes pre installed, but some of the versions of the provided packages require torch 1.6.0, which on the other hand requires python 3.6.

I tried changing the python version in the notebook via:

!sudo apt-get install python3.6
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 3 

Which then broke pip, so i needed to reinstall it. But first some packages where missing so i installed them with following commands:

!sudo add-apt-repository ppa:deadsnakes/ppa
!sudo apt update
!sudo apt-get install python3.6-distutils

After that I re-installed pip. Now the requirements.txt where running, but it failed to built some wheels during the installtion of allennlp, so the whole process was killed. Thats the point where i stopped, cause it is very tedious to get the basic notbook running.

I would appreciate some help here: am i missing something? Is there an obvious and easy way i have overseen?

I will try with conda next, but i would really appreciate help, as otherwise i have no option to train the model because i dont own an NVIDIA GPU and my CPU is 8th gen ...

EDIT: Seems like last year was the same problem, the solution provided does not work for me, as I just dont own the hardware to train locally https://github.com/sebastian-hofstaetter/teaching/discussions/69

Kind Regards,
Sebastian

[ldosen00]

Have you managed to fix the issue? I have noticed you commented on another post with similar problems?

[pintersebastian]

Hi
unfortunately, I was not able to fix this yet. I am curious if anybody else faced the same issue. If it works for other students, then i must have done something wrong...

[Joerg0]

I would expect that most of our colleagues don't read this forum as it is quite hidden at the end of the tuwel course site.

I did actually manage to install the requirements. It was pretty complicated. I provide here the baseline. I would very much like to install it on my GoogleDrive and to just reactivate it every time I do have a new runtime but I am not yet sure wether this really works.

For sure the following works within the same runtime:

from google.colab import drive
drive.mount('/content/drive')

%cd /content/drive/MyDrive/air-2023-group24/
! wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.12.0-Linux-x86_64.sh
! chmod +x Miniconda3-py37_4.12.0-Linux-x86_64.sh
! bash ./Miniconda3-py37_4.12.0-Linux-x86_64.sh -b -f -p /content/drive/MyDrive/air-2023-group24/Miniconda

!source /content/drive/MyDrive/air-2023-group24/Miniconda/bin/activate #to activate the miniconda environment

%env PYTHONPATH= '/content/drive/MyDrive/air-2023-group24/Miniconda/bin/python
!echo $PYTHONPATH

#Add miniconda to the system PATH:

import sys
sys.path.append('/content/drive/MyDrive/air-2023-group24/Miniconda/lib/python3.7/site-packages/')

import os # ?? needed?
path = '/content/drive/MyDrive/air-2023-group24/Miniconda/bin:' + os.environ['PATH']
%env PATH=$path

%cd /content/drive/MyDrive/air-2023-group24/condaENVair
!conda create --name MYcondaENVair python=3.6

%cd /content/drive/MyDrive/air-2023-group24/condaENVair
!source activate MYcondaENVair

!conda info --envs

not sure if really the desired venv is active.

!conda install --channel defaults conda python=3.6 --yes
!conda update --channel defaults --all --yes

!pip install -r /content/drive/MyDrive/air-2023-group24/requirements.txt

ATTENTION:

works for

#allennlp==1.2.2
#blingfire==0.1.7
#PyYAML==5.4
#transformers==3.5.1

WORKS NOT FOR

#torch==1.6.0

SO I DELETED THE LINE WITH torch IN THE TEXT FILE!

and installed manually:

!conda install pytorch==1.6.0 torchvision==0.7.0 -c pytorch

[Joerg0]

Well, yes - within a runtime it works. Unfortunately, reactivating the Anaconda installation from the GoogleDrive folder does not work. There is a guide that might solve it - I haven't managed to do it yet:
https://stackoverflow.com/questions/64464195/how-to-install-conda-on-google-drive-for-google-colab

[pintersebastian]

thanks a lot for your help, i got it working that way, but still...very tideous

[pintersebastian]

I guess one could put the commands into a script and store this script in drive, then just execute the script upon loading the notebook