Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Undefined environment variables are redefined in build script environment for multi-output builds #5323

Open
2 tasks done
jameslamb opened this issue Apr 30, 2024 · 0 comments · May be fixed by #5322
Open
2 tasks done

Undefined environment variables are redefined in build script environment for multi-output builds #5323

jameslamb opened this issue Apr 30, 2024 · 0 comments · May be fixed by #5322
Labels
type::bug describes erroneous operation, use severity::* to classify the type

Comments

@jameslamb
Copy link
Contributor

Checklist

  • I added a descriptive title
  • I searched open reports and couldn't find a duplicate

What happened?

"Use Environment Variables" in the conda-build docs says the following about variables in build.script_env in recipes (my emphasis)

If a listed environment variable is missing from the environment seen by the conda-build process itself, a UserWarning is emitted during the build process and the variable remains undefined.

(link)

As of #5105, conda-build does not behave that way for multi-output recipes. As of that PR, any variable mentioned in the recipe's build.script_env but not found in the environment where conda-build is called from will be set to the empty string '' in the build script environment.

For example, with a recipe like this:

# meta.yaml
package:
  name: testpkg
  version: 1.0

outputs:
  - name: test_subpackage_1
    build:
      script_env:
        - SPARKLY_UNICORN
      script:
        - python -c 'import os; print("val... " + os.environ["SPARKLY_UNICORN"])'
      requirements:
        host:
          - python

I'd expect that build script to fail if run in an environment without environment variable SPARKLY_UNICORN set.

env -u SPARKLY_UNICORN conda-build testpkg

Instead, it succeeds, because environment variable SPARKLY_UNICORN is defined and set to the empty string '' in the build script environment.

Conda Info

active environment : base
    active env location : /Users/jlamb/miniforge3
            shell level : 1
       user config file : /Users/jlamb/.condarc
 populated config files : /Users/jlamb/miniforge3/.condarc
          conda version : 23.3.1
    conda-build version : 24.1.2
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=arm64
                          __osx=14.4.1=0
                          __unix=0=0
       base environment : /Users/jlamb/miniforge3  (writable)
      conda av data dir : /Users/jlamb/miniforge3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/jlamb/miniforge3/pkgs
                          /Users/jlamb/.conda/pkgs
       envs directories : /Users/jlamb/miniforge3/envs
                          /Users/jlamb/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.3.1 requests/2.31.0 CPython/3.10.12 Darwin/23.4.0 OSX/14.4.1
                UID:GID : 501:20
             netrc file : None
           offline mode : False

Conda Config

==> /Users/jlamb/miniforge3/.condarc <==
channels:
  - conda-forge

Conda list

# packages in environment at /Users/jlamb/miniforge3:
#
# Name                    Version                   Build  Channel
attrs                     23.2.0             pyh71513ae_0    conda-forge
aws-c-auth                0.7.8                h76f1ccf_2    conda-forge
aws-c-cal                 0.6.9                hb1772db_2    conda-forge
aws-c-common              0.9.10               h93a5062_0    conda-forge
aws-c-compression         0.2.17               hb1772db_7    conda-forge
aws-c-event-stream        0.3.2                hb5e90b3_8    conda-forge
aws-c-http                0.7.14               hd747585_3    conda-forge
aws-c-io                  0.13.36              h1112932_3    conda-forge
aws-c-mqtt                0.9.10               h99ceed4_2    conda-forge
aws-c-s3                  0.4.4                hc25d294_0    conda-forge
aws-c-sdkutils            0.1.13               hb1772db_0    conda-forge
aws-checksums             0.1.17               hb1772db_6    conda-forge
awscli                    2.15.7          py310hbe9552e_0    conda-forge
awscrt                    0.19.19         py310hea2a873_2    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
boltons                   23.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310h1253130_0    conda-forge
bzip2                     1.0.8                h3422bc3_4    conda-forge
c-ares                    1.19.1               hb547adb_0    conda-forge
ca-certificates           2024.2.2             hf0a4a13_0    conda-forge
cctools                   986                  h4faf515_0    conda-forge
cctools_osx-arm64         986                  h62378fb_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h2399d43_3    conda-forge
cfgv                      3.3.1              pyhd8ed1ab_0    conda-forge
chardet                   5.2.0           py310hbe9552e_1    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.3.1          py310hbe9552e_0    conda-forge
conda-build               24.1.2          py310hbe9552e_0    conda-forge
conda-index               0.4.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     23.3.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
cryptography              40.0.2          py310hfc83b78_0    conda-forge
distlib                   0.3.8              pyhd8ed1ab_0    conda-forge
distro                    1.8.0              pyhd8ed1ab_0    conda-forge
docutils                  0.19            py310hbe9552e_1    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
fmt                       9.1.0                hffc8910_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
identify                  2.5.33             pyhd8ed1ab_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.32               pyhd8ed1ab_0    conda-forge
jsonpointer               2.0                        py_0    conda-forge
jsonschema                4.20.0                   pypi_0    pypi
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
ld64                      711                  h634c8be_0    conda-forge
ld64_osx-arm64            711                  ha4bd21c_0    conda-forge
libarchive                3.6.2                h82b9b87_1    conda-forge
libcurl                   8.2.1                hc52a3a8_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h642e427_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
liblief                   0.12.3               hb7217d7_0    conda-forge
libllvm16                 16.0.6               he79909e_2    conda-forge
libmamba                  1.4.2                h7d1d596_0    conda-forge
libmambapy                1.4.2           py310h34b6e76_0    conda-forge
libnghttp2                1.52.0               hae82a92_0    conda-forge
libsolv                   0.7.24               ha614eb4_3    conda-forge
libsqlite                 3.43.0               hb31c410_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.11.5               h25269f3_1    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
lzo                       2.10              h642e427_1000    conda-forge
mamba                     1.4.2           py310ha5d4528_0    conda-forge
markupsafe                2.1.5           py310hd125d64_0    conda-forge
menuinst                  2.0.2           py310hbe9552e_0    conda-forge
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  h7ea286d_0    conda-forge
nodeenv                   1.8.0              pyhd8ed1ab_0    conda-forge
openssl                   3.3.0                h0d3ecfb_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h27ca646_1002    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pkginfo                   1.10.0             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.1.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
pre-commit                3.6.0              pyha770c72_0    conda-forge
prompt-toolkit            3.0.38             pyha770c72_0    conda-forge
prompt_toolkit            3.0.38               hd8ed1ab_0    conda-forge
psutil                    5.9.8           py310hd125d64_0    conda-forge
py-lief                   0.12.3          py310h0f1eb42_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.4           py310h8e9501a_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.12         h01493a6_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py310hbe9552e_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py310h2aa6e3c_1    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.32.0                   pypi_0    pypi
reproc                    14.2.4               h1a8c8d9_0    conda-forge
reproc-cpp                14.2.4               hb7217d7_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               h5ef7bb8_0    conda-forge
rpds-py                   0.16.2                   pypi_0    pypi
ruamel.yaml               0.17.21         py310h8e9501a_3    conda-forge
ruamel.yaml.clib          0.2.7           py310h8e9501a_1    conda-forge
setuptools                68.1.2             pyhd8ed1ab_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.12               he1e0b03_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.3                   pypi_0    pypi
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ukkonen                   1.0.1           py310h38f39d4_4    conda-forge
urllib3                   1.26.18            pyhd8ed1ab_0    conda-forge
virtualenv                20.25.0            pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.12             pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
yaml-cpp                  0.7.0                hb7217d7_2    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstandard                 0.19.0          py310h8e9501a_0    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Additional Context

I believe this is a behavior change that was introduced recently, in #5105.
@jameslamb jameslamb added the type::bug describes erroneous operation, use severity::* to classify the type label Apr 30, 2024
@jameslamb jameslamb linked a pull request Apr 30, 2024 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type::bug describes erroneous operation, use severity::* to classify the type
Projects
🧭 Planning
Status: 🆕 New
Milestone
No milestone
1 participant
@jameslamb