Colab -> Conda -> Py3.6 and many outdated packages

[Joerg0]

After really annoying weeks and countless drawbacks we found out how to get colab doing what we want it to do for the reranking.

I would like to express my sincere thanks to the members of Group 24 and I think that they all deserve the maximum amount of possible bonus points.

Here is the code:

%env PYTHONPATH =
!wget https://repo.anaconda.com/miniconda/Miniconda3-py38_4.12.0-Linux-x86_64.sh
!chmod +x Miniconda3-py38_4.12.0-Linux-x86_64.sh
!./Miniconda3-py38_4.12.0-Linux-x86_64.sh -b -f -p /usr/local
!conda update conda -y

import sys
import os
sys.path.append("/usr/local/lib/python3.8/site-packages")
path = '/usr/local/bin:' + os.environ['PATH']
%env PATH=$path

!conda create --prefix /usr/local/envs/env_36_loc python=3.6 -y

!conda run -n env_36_loc conda info --envs # GOES

!conda run -n env_36_loc python --version

!conda run -n env_36_loc pip install -r /content/drive/MyDrive/air-2023-group24/requirements.txt

The problem is that colab behaves different wrt shell commands.
We couldn't activate the conda environment with the regular bash expressions like
!source activate "/usr/local/envs/env_36_loc" ## GOES NOT

-> Instead we had to each and every time put
!conda run -n env_36_loc
before the command we wanted to run - be it a shell command or a python script.

!conda run -n env_36_loc python your_script.py ## GOES FOR EVERY .PY FILE

Good luck!

[HackerBschor]

Also, if you want to run the code directly in a notebook cell, you can write

%%bash
source activate env_36_loc

python

### CODE ###

Unfortunately, the variables work only in the cell itself.

[Nyzabes]

Hi, first of all: thank you, this works for me in Colab!
The problem is that I only get the output of the command when the command finishes.
Is there any way to get the real-time output?
Thanks!

[Joerg0]

Actually yes, in my case it is in Colab:

%%shell
eval "$(conda shell.bash hook)"
conda activate env_36_loc
chmod +x /content/drive/MyDrive/air-2023-group24/src/re_ranking_knrm.py
python3 /content/drive/MyDrive/air-2023-group24/src/re_ranking_knrm.py

• You need first to set the execute permission on a file, allowing it to be run as a program.
  -> Then you run it and get the output in real time.

I would like to again express my sincere thanks to the great work of the members of Group 24.
The ladies there even managed to hand over certain parameters to the .py file, but for this aspect they are the experts WHO EARN CUDOS and bonus points.

[whoopi24]

I would like to again express my sincere thanks to the great work of the members of Group 24. The ladies there even managed to hand over certain parameters to the .py file, but for this aspect they are the experts WHO EARN CUDOS and bonus points.

Thank you, Joerg0 😄 It is actually not that difficult:
You have to import the package called "argparse" and define a function in your python file, e.g. get_args(), where you add the arguments which you want to have when executing the python file, see the function below:

def get_args():

    parser = argparse.ArgumentParser(description="")

    parser.add_argument(
    "-m", "--model", default="tk", choices=["tk", "knrm"], help="tk or knrm"
    )

    return parser.parse_args()

At the end of the python file, you have to add the following if clause which calls another function in the Python file when executing it:

if "__main__" == __name__:

    args = get_args()

    model_name = args.model

    evaluate_model(model_name = model_name)

In the end, you just add the argument like python3 /content/drive/MyDrive/air-2023-group24/src/re_ranking_knrm.py -m "knrm"

Best regards,
Marina

[JuLuk]

Is this also usable if i do not work in a Linux environment, but Windows?

[whoopi24]

Hi, yes that should work in both environments :)