__osx virtual package version depends on what SDK CPython in the base environment was built with

[jjhelmus]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

On the macOS of x86-64 the version of the __osx virtual package depend on what SDK CPython in the base environment was built with. Specifically it can either be >=10.16,<11.0 if CPython was built with macOS SDK <11 or >=11.0 if built with SDK >=11. This is a result of the 10.x compatibility rules that comes into play when reporting the OS version starting with Big Sur. For example see Big Sur is both 10.16 and 11.0 – it’s official.

This difference in versions for the __osx virtual packages make it difficult for package builders to determine an appropriate constraint to add when they need to describe the version of macOS their package is compatible with. This also introduced a hurdle for end users who may be blocked from installing packages that use have a dependency on __osx>=11.0.

As an example of this inconsistency:

❯ sw_vers
ProductName:		macOS
ProductVersion:		14.4.1
BuildVersion:		23E224

❯ conda create --name test_py39_defaults --platform osx-64 python=3.9 conda --yes
...
❯ ./mc/envs/test_py39_defaults/condabin/conda info | grep __osx
                          __osx=10.16=0
❯ conda create --name test_py39_cf --platform osx-64 python=3.9 conda --override-channels --channel conda-forge --yes
❯ ./mc/envs/test_py39_cf/condabin/conda info | grep __osx
                          __osx=14.4.1=0

One potential solution for this issue would be to translate the versions >=10.16,<11.0 into it's >=11.0 equivalent. This could be tricky going forward as future version are not known AFAIK.

Conda Info

❯ conda info

     active environment : base
    active env location : /Users/jhelmus/mc
            shell level : 1
       user config file : /Users/jhelmus/.condarc
 populated config files : /Users/jhelmus/.condarc
          conda version : 24.3.0
    conda-build version : 24.3.0
         python version : 3.10.13.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.3.0=0
                          __osx=14.4.1=0
                          __unix=0=0
       base environment : /Users/jhelmus/mc  (writable)
      conda av data dir : /Users/jhelmus/mc/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/jhelmus/mc/pkgs
                          /Users/jhelmus/.conda/pkgs
       envs directories : /Users/jhelmus/mc/envs
                          /Users/jhelmus/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.10.13 Darwin/23.4.0 OSX/14.4.1 solver/libmamba conda-libmamba-solver/23.11.0 libmambapy/1.5.3 aau/0.4.4 c/. s/. e/.
                UID:GID : 502:20
             netrc file : None
           offline mode : False

Conda Config

❯ conda config --show-sources
==> /Users/jhelmus/.condarc <==
add_pip_as_python_dependency: False
changeps1: False
ssl_verify: True
report_errors: False

Conda list

No response

Additional Context

No response

[jaimergp]

This is fun too:

(test_py39_defaults) $ $CONDA_PREFIX/condabin/conda info | grep __osx
                          __osx=10.16=0
(test_py39_defaults) $ SYSTEM_VERSION_COMPAT=0 $CONDA_PREFIX/condabin/conda info | grep __osx
                          __osx=14.3.1=0
(test_py39_defaults) $ SYSTEM_VERSION_COMPAT=1 $CONDA_PREFIX/condabin/conda info | grep __osx
                          __osx=10.16=0

So maybe it's a matter of setting the right env var? This env var needs to be set up before the Python process starts, so we will need a subprocess for this to work reliably.

[jjhelmus]

Following the virtual package path the value comes from platform.mac_ver()[0].
In turn this is determined by reading the version from a plist file in platform._mac_ver_xml.

One option would be to this read this file via a subprocess with SYSTEM_VERSION_COMPAT=0 set:

def non_compat_mac_ver() -> Optional[str]:
    # adapted from _mac_ver_xml in the stdlib platform module
    fn = '/System/Library/CoreServices/SystemVersion.plist'
    if not os.path.exists(fn):
        if 'SDKROOT' in os.environ:
            fn = os.environ['SDKROOT'] + fn
            if not os.path.exists(fn):
                return None
        else:
            return None
    result = subprocess.run(
        ["/bin/cat", fn],
        capture_output=True,
        env = {"SYSTEM_VERSION_COMPAT": "0"}
    )
    pl = plistlib.loads(result.stdout)
    release = pl['ProductVersion']
    return release

This returns the non-compatible version:

❯ cat example.py 
... 
import platform
orig_ver = platform.mac_ver()[0]
non_compat_ver = non_compat_mac_ver()
print(f"Original version: {orig_ver}")
print(f"Non-compat version: {non_compat_ver}")

❯ python example.py 
Original version: 10.16
Non-compat version: 14.4.1

[jjhelmus]

Another option would be to subprocess out to python with the environment variable set:

def non_compat_mac_ver() -> Optional[str]:
    result = subprocess.run(
        [
            sys.executable,
            "-c",
            "import platform; print(platform.mac_ver()[0])"
        ],
        env = {"SYSTEM_VERSION_COMPAT": "0"},
        capture_output=True,
        encoding="utf-8",
    )
    return result.stdout