Unable to run integration tests locally due to self-signed certificate error

[briantist]

$ git clone https://github.com/hvac/hvac.git
$ cd hvac
$ poetry install
$ poetry shell
$ make test
pytest --cov=hvac tests/
make: *** [Makefile:7: test] Error 1

$ pytest tests -x
<snip>
                except Exception as ex:
                    print(ex)
                    if process.poll() is not None:
>                       raise Exception("Vault server terminated before becoming ready")
E                       Exception: Vault server terminated before becoming ready

tests/utils/server_manager.py:74: Exception
---------------------------------------------------------------------------------------------- Captured stdout setup -----------------------------------------------------------------------------------------------
HTTPSConnectionPool(host='localhost', port=8200): Max retries exceeded with url: /v1/sys/init (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)')))
HTTPSConnectionPool(host='localhost', port=8200): Max retries exceeded with url: /v1/sys/init (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1006)')))
================================================================================================= warnings summary =================================================================================================
tests/utils/__init__.py:10
  /home/loot/git/hvac/tests/utils/__init__.py:10: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils.spawn import find_executable

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

I had to add a print() statement to actually see the exception that was causing it to fail.

I looked at the CI workflows, but those don't do any certificate magic or anything, so I'm at a loss as to why it doesn't work on my machine. I'm using HVAC with mTLS in one of my projects with no issues, so I suspect the certificates aren't being loaded during testing. Currently using Python 3.11.5 on Arch Linux, if that helps.

Originally posted by @dosisod in #1049 (comment)

[briantist]

@dosisod thanks for raising this! I did some cursory investigation but couldn't find anything just yet so I split it off to a new issue to track it.

[dosisod]

Oh, silly me, it was failing because I was already running Vault in another terminal window. Turned on my computer today and the tests passed with no issue, then I realized yesterday I was running Vault to verify that my changes worked. Since I was using mTLS, the self signed certificate error threw me off.

Perhaps we could add a "make sure Vault isn't already running before testing" warning to the docs, or use a PID file to ensure the Vault server was created by the test suite? There are probably cases where you want to run tests against your own Vault server, so in that case there would have to be an env var flag to disable that.

[briantist]

Thanks for following up @dosisod . Maybe the quickest change we could make there is choosing non-standard ports. I think we already even have a helper function in the project for finding a free port but we use for other things (LDAP?).

I think most of the tests are destructive so there probably isn't a case where you'd want to run them against a Vault server that's used for anything else, but I have had it in the back of my mind to decouple starting Vault from the tests themselves, that way we could allow for multiple ways of providing the Vault server to test with, which opens up using containers or other options. That's what we do in the community.hashi_vault ansible collection's tests for example.