Test suite fails collection due to svn check

[jaraco]

Environment

• pip version: 720c77a
• Python version: 3.8.1
• OS: macOS Catalina 10.15.4

Description

This week, I upgraded to macOS 10.15.4. Now, when I run svn --version, I see this error:

pip master $ svn --version                                                                                                                                                
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.

When I run the pip test suite, I see similar errors during test discovery, which causes the whole test suite not to run.

Expected behavior

Tests for svn really shouldn't block tests.

How to Reproduce

Presumably:

1. Update to macOS 10.15.4.
2. Run tox.

Output

$ tox
GLOB sdist-make: /Users/jaraco/code/public/pypa/pip/setup.py
python inst-nodeps: /Users/jaraco/code/public/pypa/pip/.tox/.tmp/package/1/pip-20.1.dev0.zip
python installed: apipkg==1.5,atomicwrites==1.3.0,attrs==19.3.0,cffi==1.14.0,coverage==5.0.4,cryptography==2.8,csv23==0.2,execnet==1.7.1,freezegun==0.3.15,mock==4.0.2,more-itertools==8.2.0,pip==20.1.dev0,pluggy==0.13.1,pretend==1.0.9,py==1.8.1,pycparser==2.20,pytest==3.8.2,pytest-cov==2.8.1,pytest-forked==1.1.3,pytest-rerunfailures==6.0,pytest-timeout==1.3.4,pytest-xdist==1.27.0,python-dateutil==2.8.1,PyYAML==5.3.1,scripttest==1.3,setuptools==41.2.0,six==1.14.0,virtualenv==16.7.10,Werkzeug==0.16.0,wheel==0.34.2
python run-test-pre: PYTHONHASHSEED='1630556793'
python run-test-pre: commands[0] | python -c 'import shutil, sys; shutil.rmtree(sys.argv[1], ignore_errors=True)' /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels
python run-test-pre: commands[1] | python /Users/jaraco/code/public/pypa/pip/tools/tox_pip.py wheel -w /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels -r /Users/jaraco/code/public/pypa/pip/tools/requirements/tests-common_wheels.txt
Collecting setuptools>=40.8.0
  Using cached setuptools-46.1.3-py3-none-any.whl (582 kB)
  Saved ./tests/data/common_wheels/setuptools-46.1.3-py3-none-any.whl
Collecting wheel
  Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
  Saved ./tests/data/common_wheels/wheel-0.34.2-py2.py3-none-any.whl
Skipping setuptools, due to already being wheel.
Skipping wheel, due to already being wheel.
python run-test: commands[0] | pytest --timeout 300
============================= test session starts ==============================
platform darwin -- Python 3.8.1, pytest-3.8.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jaraco/code/public/pypa/pip, inifile: setup.cfg
plugins: timeout-1.3.4, rerunfailures-6.0, forked-1.1.3, cov-2.8.1, xdist-1.27.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 1533 items / 7 errors

==================================== ERRORS ====================================
_______________ ERROR collecting tests/functional/test_freeze.py _______________
tests/functional/test_freeze.py:175: in <module>
    def test_freeze_svn(script, tmpdir):
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
______________ ERROR collecting tests/functional/test_install.py _______________
tests/functional/test_install.py:272: in <module>
    def test_basic_install_editable_from_svn(script):
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
____________ ERROR collecting tests/functional/test_install_reqs.py ____________
tests/functional/test_install_reqs.py:116: in <module>
    def test_multiple_requirements_files(script, tmpdir):
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
____________ ERROR collecting tests/functional/test_install_user.py ____________
tests/functional/test_install_user.py:28: in <module>
    class Tests_UserSite:
tests/functional/test_install_user.py:48: in Tests_UserSite
    def test_install_subversion_usersite_editable_with_distribute(
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
_____________ ERROR collecting tests/functional/test_uninstall.py ______________
tests/functional/test_uninstall.py:325: in <module>
    def test_uninstall_editable_from_svn(script, tmpdir):
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
___________ ERROR collecting tests/functional/test_uninstall_user.py ___________
tests/functional/test_uninstall_user.py:8: in <module>
    from tests.functional.test_install_user import _patch_dist_in_site_packages
<frozen importlib._bootstrap>:991: in _find_and_load
    ???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:655: in _load_unlocked
    ???
<frozen importlib._bootstrap>:618: in _load_backward_compatible
    ???
.tox/python/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:290: in load_module
    six.exec_(co, mod.__dict__)
tests/functional/test_install_user.py:28: in <module>
    class Tests_UserSite:
tests/functional/test_install_user.py:48: in Tests_UserSite
    def test_install_subversion_usersite_editable_with_distribute(
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
___________________ ERROR collecting tests/unit/test_vcs.py ____________________
tests/unit/test_vcs.py:415: in <module>
    def test_subversion__call_vcs_version():
tests/lib/__init__.py:1112: in need_svn
    return pytest.mark.svn(need_executable(
tests/lib/__init__.py:1081: in wrapper
    subprocess.check_output(check_cmd)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:411: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py:512: in run
    raise CalledProcessError(retcode, process.args,
E   subprocess.CalledProcessError: Command '('svn', '--version')' returned non-zero exit status 72.
------------------------------- Captured stderr --------------------------------
svn: error: Failed to locate 'svn'.
svn: error: The subversion command line tools are no longer provided by Xcode.
=========================== short test summary info ============================
ERROR tests/functional/test_freeze.py
ERROR tests/functional/test_install.py
ERROR tests/functional/test_install_reqs.py
ERROR tests/functional/test_install_user.py
ERROR tests/functional/test_uninstall.py
ERROR tests/functional/test_uninstall_user.py
ERROR tests/unit/test_vcs.py
!!!!!!!!!!!!!!!!!!! Interrupted: 7 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 7 error in 7.96 seconds ============================
ERROR: InvocationError for command /Users/jaraco/code/public/pypa/pip/.tox/python/bin/pytest --timeout 300 (exited with code 2)
___________________________________ summary ____________________________________
ERROR:   python: commands failed

[jaraco]

I also updated XCode to 11.4 this week, so it may be that update and not macOS update that's implicated.

[jaraco]

Release notes for XCode 11 say:

Command line support for Subversion will be removed in a future release. (50195246, 50231958, 50266910, 51740851, 52528748)

[jaraco]

Seems others are confused too.

[uranusjr]

The test suite already has functionality to skip Mercurial tests if it’s not available. We probably need to do the same (or improve it) for this.

[gutsytechster]

The test suite also does the same for svn tests. Could this be happening because it only catches OSError? And from the traceback given in the issue, it seems that it throws a subprocess.CalledProcessError.

If yes, then, should catching this exception would resolve the issue?

[uranusjr]

@gutsytechster Yes, that sounds right to me.

[gutsytechster]

Great! I'd like to open a PR for it then. :)