Long conda info error messages for common exceptions

[jaimergp]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

Originally posted by @h-vetinari in conda-forge/conda-forge-ci-setup-feedstock#296

---

After the switch to conda-build (in v4 of the ci setup), something that happens very regularly on feedstocks -- a failing build script -- will now (occasionally?) cause long error reports from conda, which make it more confusing where the error is coming from. Here's an example. Everything but the first and last two lines should not be there.

  ERROR: Failed building wheel for mesas
ERROR: Could not build wheels for mesas, which is required to install pyproject.toml-based projects

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "C:\Miniforge\lib\site-packages\conda\exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
      File "C:\Miniforge\lib\site-packages\conda\cli\main.py", line 83, in main_subshell
        exit_code = do_call(args, parser)
      File "C:\Miniforge\lib\site-packages\conda\cli\conda_argparse.py", line 140, in do_call
        result = plugin_subcommand.action(getattr(args, "_args", args))
      File "C:\Miniforge\lib\site-packages\conda_build\plugin.py", line 10, in build
        execute(*args, **kwargs)
      File "C:\Miniforge\lib\site-packages\conda_build\cli\main_build.py", line 568, in execute
        outputs = api.build(
      File "C:\Miniforge\lib\site-packages\conda_build\api.py", line 253, in build
        return build_tree(
      File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3804, in build_tree
        packages_from_this = build(
      File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 2629, in build
        windows.build(
      File "C:\Miniforge\lib\site-packages\conda_build\windows.py", line 344, in build
        check_call_env(
      File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 445, in check_call_env
        return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
      File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 421, in _func_defaulting_env_to_os_environ
        raise subprocess.CalledProcessError(proc.returncode, _args)
    subprocess.CalledProcessError: Command '['cmd.exe', '/d', '/c', 'conda_build.bat']' returned non-zero exit status 1.

`$ C:\Miniforge\Scripts\conda-script.py build recipe -m .ci_support\win_64_python3.9.____cpython.yaml --suppress-variables --extra-meta flow_run_id=azure_20231108.1.2 remote_url=*** sha=dadb1c68ff0ffc1af148ec1922f6943671dfea26`

  environment variables:
BUILD_DEFINITIONFOLDERPATH=\
BUILD_REPOSITORY_LOCALPATH=D:\a\1\s
                 CIO_TEST=<not set>
                    CONDA=C:\Miniconda
    CONDA_ALLOW_SOFTLINKS=false
           CONDA_BLD_PATH=D:\\bld\\
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=C:\Miniforge\condabin\..\Scripts\conda.exe
CONDA_LIBMAMBA_SOLVER_NO_CHANNELS_FROM_INSTALLED=1
             CONDA_PREFIX=C:\Miniforge
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=C:\Miniforge\python.exe
               CONDA_ROOT=C:\Miniforge
              CONDA_SHLVL=1
             CONDA_SOLVER=libmamba
           CURL_CA_BUNDLE=<not set>
           ENTRYPOINTPATH=C:\Miniforge\Scripts\SetPageFileSize.ps1
                 HOMEPATH=\Users\VssAdministrator
               LD_PRELOAD=<not set>
                     PATH=C:\Miniforge;C:\Miniforge\Library\mingw-
                          w64\bin;C:\Miniforge\Library\usr\bin;C:\Miniforge\Library\bin;C:\Minif
                          orge\Scripts;C:\Miniforge\bin;C:\Miniforge\condabin;C:\Miniforge\Scrip
                          ts;C:\agents\3.230.0\externals\git\cmd;C:\agents\3.230.0\externals\git
                          \mingw64\bin;C:\PROGRA~1\MongoDB\bin;C:\aliyun-cli;C:\vcpkg;C:\Program
                          Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\h\windows
                          \stack\2.13.1\x64;C:\cabal\bin;C:\ghcup\bin;C:\mingw64\bin;C:\Program
                          Files\dotnet;C:\Program Files\MySQL\MySQL Server 8.0\bin;C:\Program
                          Files\R\R-4.3.2\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program
                          Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)
                          \pipx_bin;C:\npm\prefix;C:\h\windows\go\1.20.11\x64\bin;C:\h\windows\P
                          ython\3.9.13\x64\Scripts;C:\h\windows\Python\3.9.13\x64;C:\h\windows\R
                          uby\3.0.6\x64\bin;C:\tools\kotlinc\bin;C:\h\windows\Java_Temurin-
                          Hotspot_jdk\8.0.392-8\x64\bin;C:\Program
                          Files\ImageMagick-7.1.1-Q16-HDRI;C:\Program Files\Microsoft SDKs\Azure
If submitted, this report will be used by core maintainers to improve
                          \CLI2\wbin;C:\ProgramData\kind;C:\Windows\system32;C:\Windows;C:\Windo
future releases of conda.
                          ws\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows
                          \System32\OpenSSH;C:\Program Files\dotnet;C:\Program
                          Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform
                          Installer;C:\Program Files\Microsoft SQL Server\Client
                          SDK\ODBC\170\Tools\Binn;C:\Program Files\Microsoft SQL
                          Server\150\Tools\Binn;C:\Program Files (x86)\Windows Kits\10\Windows
                          Performance Toolkit;C:\Program Files\Microsoft SQL
                          Server\130\DTS\Binn;C:\Program Files\Microsoft SQL
                          Server\140\DTS\Binn;C:\Program Files\Microsoft SQL
                          Server\150\DTS\Binn;C:\Program Files\Microsoft SQL Server\160\DTS\Binn
                          ;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program
                          Files\TortoiseSVN\bin;C:\ProgramData\chocolatey\lib\maven\apache-
                          maven-3.8.7\bin;C:\Program Files\Microsoft Service
                          Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service
                          Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program
                          Files\nodejs;C:\Program Files\GitHub CLI;C:\SeleniumWebDrivers\ChromeD
                          river;C:\SeleniumWebDrivers\EdgeDriver;C:\Program
                          Files\Amazon\AWSCLIV2;C:\Program
                          Files\Amazon\SessionManagerPlugin\bin;C:\Program
                          Files\Amazon\AWSSAMCLI\bin;C:\Program Files\Microsoft SQL
                          Server\130\Tools\Binn;C:\Program Files\LLVM\bin;C:\Users\VssAdministra
                          tor\.dotnet\tools;C:\Users\VssAdministrator\.cargo\bin;C:\Users\VssAdm
                          inistrator\AppData\Local\Microsoft\WindowsApps
PSMODULEANALYSISCACHEPATH=C:\PSModuleAnalysisCachePath\ModuleAnalysisCache
             PSMODULEPATH=C:\Users\VssAdministrator\Documents\WindowsPowerShell\Modules;C:\\Modu
                          les\azurerm_2.1.0;C:\\Modules\azure_2.1.0;C:\Users\packer\Documents\Wi
                          ndowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\
                          Windows\system32\WindowsPowerShell\v1.0\Modules;C:\Program
                          Files\Microsoft SQL Server\130\Tools\PowerShell\Modules\
         PYTHONUNBUFFERED=1
       REQUESTS_CA_BUNDLE=<not set>
        SELENIUM_JAR_PATH=C:\selenium\selenium-server.jar
            SSL_CERT_FILE=<not set>

     active environment : base
    active env location : C:\Miniforge
            shell level : 1
       user config file : C:\Users\VssAdministrator\.condarc
 populated config files : C:\Miniforge\.condarc
                          C:\Users\VssAdministrator\.condarc
          conda version : 23.10.0
    conda-build version : 3.27.0
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __win=0=0
       base environment : C:\Miniforge  (writable)
      conda av data dir : C:\Miniforge\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : C:\Miniforge\pkgs
                          C:\Users\VssAdministrator\.conda\pkgs
                          C:\Users\VssAdministrator\AppData\Local\conda\conda\pkgs
       envs directories : C:\Miniforge\envs
                          C:\Users\VssAdministrator\.conda\envs
                          C:\Users\VssAdministrator\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.10.0 requests/2.31.0 CPython/3.10.12 Windows/10 Windows/10.0.20348 solver/libmamba conda-libmamba-solver/23.11.1 libmambapy/1.5.3
          administrator : True
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.
If you suspect this error is being caused by a malfunctioning plugin,
consider using the --no-plugins option to turn off plugins.

Example: conda --no-plugins install <package>

Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
the command line to run the command without plugins enabled.

Example: CONDA_NO_PLUGINS=true conda install <package>


No report sent. To permanently opt-out, use

    $ conda config --set report_errors false

##[error]Cmd.exe exited with code '1'.
##[section]Finishing: Run Windows build

CC @jaimergp, who asked me to open this issue after it came up in discussion in last week's core call.

Conda Info

See above

Conda Config

N/A

Conda list

N/A

Additional Context

@mbargull shared more details in the Element chat:

We applied conda-forge/conda-smithy#1859 for conda-forge's builds.
Essentially, it's conda-build non-clean exits via raising exceptions (e.g., CalledProcessError in case a build script fails) and then conda handling those exceptions with its internal exception handler if conda build is run (i.e., not conda-build directly but via subcommand plugin).
Essentially we need 2 issues (in case there aren't any already):

1. issue for conda-build to more cleanly exit in error cases,
2. issue for conda to better handle non-internal exceptions from subcommand plugins.

Oh, and if you need a test case for this, then a simple recipe with a failing build script should suffice, e.g.,

./meta.yaml cat << 'EOF'
package:
 name: pkg
 version: 1.0
build:
 script: exit 1
EOF
conda build .

[jaimergp]

This is a reproducer test with current conda and conda-build 24.3.0:

def test_conda_build_errors_without_conda_info_handlers(tmp_path, testing_workdir):
    ""
    recipe = tmp_path / "meta.yaml"
    recipe.write_text(
        dedent(
            """
            package:
                name: pkg
                version: '1.0'
            build:
                script: exit 1
            """
        )
    )
    process = subprocess.run(["conda", "build", tmp_path], capture_output=True, text=True, check=False)
    assert process.returncode > 0
    all_output = process.stdout + "\n" + process.stderr

    # These should NOT appear in the report
    assert ">>> ERROR REPORT <<<" not in all_output
    assert "An unexpected error has occurred." not in all_output
    assert "Conda has prepared the above report." not in all_output

    # These should appear
    assert "CalledProcessError" in all_output
    assert "returned non-zero exit status 1" in all_output

After looking into it, I see several ways forward:

• Make conda-build raise exceptions that are children of conda.exceptions.CondaError. This will involve a few changes all around the codebase.
• Make conda-build raise exceptions that are children of conda_build.exceptions.CondaBuildError and then have the conda plugin system recognize that as a known exception type.
• Do (2) and also add a general mechanism in conda so plugins can define which base class their exceptions will be based on, so conda error reporting doesn't panic that loudly.

Something like:

  @conda.plugins.hookimpl
  def conda_subcommands():
      yield conda.plugins.CondaSubcommand(
          name="build",
          summary="Build conda packages from a conda recipe.",
          action=build,
+         base_exception_class=CondaBuildError,
      )

Then the plugin manager will be able to collect those and expose them to the exception handler so they can be checked before falling back to handler_unexpected_exception.

WDYT @conda/conda-core?