Using Python_jll with PyCall

[marius311]

Is there a reason there's not an easy way to use Python_jll's Python with PyCall, something like ENV["PYTHON"]="Python_jll" or something?

As it stands to get the linking right you can do

LD_LIBRARY_PATH=$(julia -e "using Python_jll; print(Python_jll.LIBPATH[])") \
    PYTHON=$(julia -e "using Python_jll; print(Python_jll.python_path)") \
    julia -e 'using Pkg; Pkg.build("PyCall")'

and then you need using Python_jll, PyCall but this could presumably be automated here.

Would a PR adding this be something of interest?

[stevengj]

I'm not opposed in principle, but how useful is that? How easy it is to install Python packages into Python_jll?

[marius311]

Yea I think thats indeed the key thing deciding usefulness. Fwiw its pretty easy to do poetry env use $(julia --startup-file=no -e "using Python_jll; print(Python_jll.python_path)") then install anything in that virtual environment, but the dynamic linking isnt 100% right unless you manually set LD_LIBRARY_PATH (theres tantalizingly just one single library, libffi, that isnt on an RPATH). I'm still playing with this set up.

In terms of a PR, what I realize is I'm not sure you can do it without having PyCall depend on Python_jll, which seems unwanted. Its because to get the linking right on the Julia side, you need to load Python_jll in PyCall's init. But PyCall cant do that unless its a dep? I guess you could just assume eval Main Python_jll will work? (still seems not great)

[stevengj]

Because we generate a deps.jl file, you wouldn't have to use eval, but could instead put it as an optional dependency in Julia 1.9?