Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Side-effects from .condarc: packages added from create_default_packages #492

Open
2 tasks done
calebzulawski opened this issue Aug 28, 2023 · 10 comments
Open
2 tasks done

Comments

@calebzulawski
Copy link

calebzulawski commented Aug 28, 2023

Original title: channels field in .condarc doesn't seem to be respected

Checklist

  • I added a descriptive title
  • I searched open reports and couldn't find a duplicate

What happened?

I have a .condarc similar to:

channels:
  - conda-forge
create_default_packages:
  - python

My environment spec doesn't have channels.

When I use conda-lock, I need to add -c conda-forge despite being in my condarc. I know my condarc is getting used because the create_default_packages contents end up in the lockfile.

Conda Info

I'm using micromamba: 


           environment : None (not found)
           env location : -
      user config files : /home/caleb/.mambarc
 populated config files : 
       libmamba version : 1.4.3
     micromamba version : 1.4.3
           curl version : libcurl/7.88.1 OpenSSL/3.1.0 zlib/1.2.13 zstd/1.5.2 libssh2/1.10.0 nghttp2/1.52.0
     libarchive version : libarchive 3.6.2 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.2
       virtual packages : __unix=0=0
                          __linux=5.19.0=0
                          __glibc=2.35=0
                          __archspec=1=x86_64
               channels : 
       base environment : /home/caleb/micromamba
               platform : linux-64


### Conda Config

```shell
Configuration files (by precedence order):
./.condarc


### Conda list

```shell
I'm not sure the micromamba equivalent

Additional Context

No response

@maresb
Copy link
Contributor

maresb commented Aug 29, 2023

Thanks a lot for this bug report. This is very interesting.

I'd actually argue that conda-lock picking up the create_default_packages is a bug. The reason is that conda-lock is supposed to transform a specification into a lockfile. Mixing up the specification and the local .condarc config is going to lead to problems and confusion where different computers produce different lockfiles, and the reason being very difficult to track down.

@mfisher87
Copy link
Contributor

I'd actually argue that conda-lock picking up the create_default_packages is a bug. The reason is that conda-lock is supposed to transform a specification into a lockfile. Mixing up the specification and the local .condarc config is going to lead to problems and confusion where different computers produce different lockfiles, and the reason being very difficult to track down.

💯

This aspect of conda-lock's reproducibility is IMO really important, but not clearly enough communicated in the README or docs. The docs are really clear that conda-lock's objective is to record enough information so that a conda environment can be created twice identically (excepting packages being removed from repositories out of its control), but not so clear about the advantages of eliminating (excepting changes in repository metadata out of its control) side effects in the solving process itself.

What you said above does communicate that clearly, and it feels like it would be valuable to copy it directly in to the docs :)

@calebzulawski
Copy link
Author

calebzulawski commented Aug 29, 2023

Perhaps what I need for my use-case is to transform env.yaml to include channels and default packages, which should have a similar result. I could also see this as similar to environment variables however, where the spec is not complete without knowing the values. I think you could equally include default channel and package information from the .condarc in the lockfile if necessary.

@maresb
Copy link
Contributor

maresb commented Aug 29, 2023

Is there anything about your use case that makes explicitly including everything in the env.yaml cumbersome?

@calebzulawski
Copy link
Author

It should be possible--in my use case it happens that the condarc is controlled by a build system and the dependencies are defined by the user, so that's why I care about the distinction.

@maresb
Copy link
Contributor

maresb commented Aug 29, 2023

Ah, got it.

You know, I wouldn't be opposed to adding .condarc as a spec source in the sense of --file=~/.condarc.

@calebzulawski
Copy link
Author

I think that would be perfect, assuming you could provide the condarc along with a spec! I actually tried that first before submitting this issue, on a whim that it might work.

@maresb
Copy link
Contributor

maresb commented Aug 29, 2023

I don't have the bandwidth to work on this. Would you like to write a src_parser for .condarc and open a PR?

@maresb maresb changed the title channels field in .condarc doesn't seem to be respected Side-effects from .condarc: packages added from create_default_packages Aug 29, 2023
@calebzulawski
Copy link
Author

I'll take a look into it!

@maresb
Copy link
Contributor

maresb commented Aug 29, 2023

Great! And if you see things that seem unnecessarily complicated or lacking in documentation, then if you're feeling up to it please be bold about making changes to clean things up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants