Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix create_eagle_env.sh #173

Closed
wants to merge 2 commits into from
Closed

Conversation

rHorsey
Copy link
Contributor

@rHorsey rHorsey commented Aug 5, 2020

Fix for module update that breaks the source command for anaconda on Eagle.

Pull Request Description

In the latest eagle system update source activate /path/to/env/here no longer correctly works. This leads to problems with both the documentation and the create_eagle_env.sh script.

Proposed migration text:

This version updates documentation to fix breakages introduced by the system image (and associated module libraries) on eagle, specifically the source activate command no longer working. conda activate is now required instead place.

Checklist

Not all may apply

  • Code changes (must work)
  • Tests exercising your feature/bug fix (check coverage report on CircleCI build -> Artifacts)
  • All other unit tests passing
  • Update validation for project config yaml file changes
  • Update existing documentation
  • Run a small batch run to make sure it all works (local is fine, unless an Eagle specific feature)
  • Add to the changelog_dev.rst file and propose migration text in the pull request

@rHorsey rHorsey added bug Something isn't working eagle needs review labels Aug 5, 2020
@rHorsey rHorsey requested a review from nmerket August 5, 2020 20:49
@rHorsey
Copy link
Contributor Author

rHorsey commented Aug 5, 2020

@mpraprost Is testing on eagle presently.

@rHorsey
Copy link
Contributor Author

rHorsey commented Aug 5, 2020

Oh goody - on non-login nodes source activate is required and conda activate does not work, which explains why the other eagle shell scripts are working correctly. @nmerket

@nmerket
Copy link
Member

nmerket commented Aug 5, 2020

Wow. Just wow. Maybe we should contact hpc-help about this and see what their recommended way of working with conda is.

@mpraprost
Copy link

mpraprost commented Aug 6, 2020

Recreated the error:

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting pip
Using cached https://files.pythonhosted.org/packages/bd/b1/56a834acdbe23b486dea16aaf4c27ed28eb292695b90d01dff96c96597de/pip-20.2.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.3.1
Uninstalling pip-19.3.1:
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/nopt/nrel/apps/anaconda/5.3/bin/pip'
Consider using the --user option or check the permissions.

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Obtaining file:///lustre/eaglefs/projects/comstock/repos/covid-bsb
ERROR: Package 'buildstock-batch' requires a different Python: 2.7.15 not in '>=3.6'

@rHorsey
Copy link
Contributor Author

rHorsey commented Aug 6, 2020

So - this error is now intermittent. Sometimes we hit it with source activate and sometimes with conda activate - in either case the issue manifests as python 2.7 being loaded from the path and then various errors ensue.

@rHorsey rHorsey added DO NOT MERGE PR is open but please don't merge! invalid This doesn't seem right and removed needs review labels Aug 6, 2020
@nmerket
Copy link
Member

nmerket commented Aug 10, 2020

This is not a new issue. I can't remember exactly but depending on how things are module loaded and how environments are loaded after that, there can be issues. Both of those operations mess with the PATH and can conflict with each other. I think you have to do those operations in order. Like you have to module load conda and then source activate /path/to/env.

@nmerket
Copy link
Member

nmerket commented May 4, 2021

We tracked down the problem in #226.

@nmerket nmerket closed this May 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working DO NOT MERGE PR is open but please don't merge! eagle invalid This doesn't seem right
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants