Help needed with SSL certificates

[egavazzi]

I am calling a Python package from Julia using PythonCall. A function in the package is doing an HTTP request to download some data from a website. When I import and run the function from Julia, I get the following error about some SSL certificate issue.

ERROR: Python: URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)>

However, when I run the same function directly from the Python installed in the local conda environment (the .CondaPkg env), everything is working like a charm. Is there something I am doing wrong? Maybe there are some option I should change in PythonCall related to libraries or something? I am a bit desperate right now after trying some solutions for hours without any success so I am hoping someone more competent will be able to help 😅

[cjdoris]

Only thing that springs to mind is to see if os.environ is different in PythonCall versus directly in Python (in the same Conda environment). If so, as a workaround try setting the appropriate environment vars in Julia before loading PythonCall. Let me know.

[cjdoris]

Calling the underlying C functions that get the cert paths:

First just with OpenSSL:

julia> using OpenSSL: libssl

julia> ccall((:X509_get_default_cert_file, libssl), Cstring, ())
Cstring(0x00007f0a593ca018)

julia> unsafe_string(ans)
"/workspace/destdir/ssl/cert.pem"

Now loading Python ssl first:

julia> using PythonCall

julia> @py import ssl

julia> ssl.get_default_verify_paths()
Python: DefaultVerifyPaths(cafile='/home/cjdoris/.julia/environments/python/.CondaPkg/env/ssl/cert.pem', capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/home/cjdoris/.julia/environments/python/.CondaPkg/env/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/home/cjdoris/.julia/environments/python/.CondaPkg/env/ssl/certs')

julia> using OpenSSL: libssl

julia> ccall((:X509_get_default_cert_file, libssl), Cstring, ())
Cstring(0x00007fb6d9e119a8)

julia> unsafe_string(ans)
"/home/cjdoris/.julia/environments/python/.CondaPkg/env/ssl/cert.pem"

This is all consistent with the above findings. I guess really it's just showing that the unwanted default paths really are coming from the version of libssl loaded by OpenSSL.

Python ssl is hard-coded to use X509_get_default_cert_file to get the cert file, and that path is hard-coded into libssl. Presumably the version of libssl distributed by Python/Conda is set-up to direct to the cert file distributed with Python/Conda, whereas the version of libssl distributed by OpenSSL.jl (namely OpenSSL_jll) does not do this.

[egavazzi]

Ah yes I didn't think about trying different orders when checking libssl. I don't know if forcing a newer version could fix the issue, but I found that setting the environment variable SSL_CERT_FILE before importing ssl (in the case we already loaded OpenSSL) makes it work. I set it to the output of NetworkOptions.ca_roots(), which according to the docstring returns the path to the file where the root certificates of the system are stored.

using OpenSSL
using NetworkOptions
using PythonCall

ENV["SSL_CERT_FILE"] = NetworkOptions.ca_roots()
request = pyimport("urllib.request")
request.urlopen("https://celestrak.org/SpaceData/SW-All.csv")

[egavazzi]

I didn't see your newest comment when answering. Thank you for your findings and explanation (I am not sure I would have found these things with X509_get_default_cert_file myself, or at least it would have taken a lot more time).
However I don't really understand the cert path returned by libssl used by OpenSSL (/workspace/destdir/ssl/cert.pem) and why this works with HTTP.jl but not with urllib.

[cjdoris]

I think HTTP.jl just doesn't use OpenSSL's defaults at all and uses NetworkOptions instead to find the certs.

There may be some magic PythonCall can do to get this working automagically, for example it could set SSL_CERT_FILE as you do above, or could do

ssl.create_default_context.__kwdefaults__["cafile"] = NetworkOptions.ca_roots()

[egavazzi]

Nice, I didn't know one could set default keyword arguments this way. Both way work indeed. Thank you very much for your help on this issue, I will mark your comment as answer to the question 👍