SymPy cannot be loaded properly after updating the Julia to a new version

[songhanzhang]

Recently, I updated my Julia to a new version:

Julia Version 1.9.1
Commit 147bdf428cd (2023-06-07 08:27 UTC)
Platform Info:
OS: macOS (arm64-apple-darwin22.4.0)
CPU: 8 × Apple M1
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
Threads: 8 on 4 virtual cores
Environment:
JULIA_EDITOR = atom -a
JULIA_NUM_THREADS = 8

The package "SymPy" cannot be used properly. The error messages are given below. (The package remains work properly when using an old version Julia 1.7.3 on another computer.)

julia> using SymPy
[ Info: Precompiling SymPy [24249f21-da20-56a4-8eb1-6a02cf4ae2e6]
Python path configuration:
PYTHONHOME = '/Users/songhan.zhang/.julia/conda/3:/Users/songhan.zhang/.julia/conda/3'
PYTHONPATH = (not set)
program name = '/Users/songhan.zhang/.julia/conda/3/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.base_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.base_exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.platlibdir = 'lib'
sys.executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.path = [
'/Users/songhan.zhang/.julia/conda/3/lib/python310.zip',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000001fdbd9e00 (most recent call first):

ERROR: Failed to precompile SymPy [24249f21-da20-56a4-8eb1-6a02cf4ae2e6] to "/Users/songhan.zhang/.julia/compiled/v1.9/SymPy/jl_R5cd1i".
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base ./loading.jl:2296
[3] compilecache
@ ./loading.jl:2163 [inlined]
[4] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1805
[5] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1660
[6] macro expansion
@ ./loading.jl:1648 [inlined]
[7] macro expansion
@ ./lock.jl:267 [inlined]
[8] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1611

[jverzani]

I'm no expert, but it looks like your python isn't full configured. Can you try this?

using PyCall
sympy = pyimport_conda("sympy", "sympy")
sympy.sin

My guess is either the first or second line is going to be an issue, and my guess is the first.

[songhanzhang]

I'm no expert, but it looks like your python isn't full configured. Can you try this?

using PyCall
sympy = pyimport_conda("sympy", "sympy")
sympy.sin

My guess is either the first or second line is going to be an issue, and my guess is the first.

Thanks for your response! Yes, your prediction is exactly correct. However, I never changed the Python configurations (as I never use Python) except for updating the Julia version.

julia> using PyCall
Python path configuration:
PYTHONHOME = '/Users/songhan.zhang/.julia/conda/3:/Users/songhan.zhang/.julia/conda/3'
PYTHONPATH = (not set)
program name = '/Users/songhan.zhang/.julia/conda/3/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.base_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.base_exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.platlibdir = 'lib'
sys.executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.path = [
'/Users/songhan.zhang/.julia/conda/3/lib/python310.zip',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000000203001e00 (most recent call first):

Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]

[jverzani]

Maybe Pkg.build("PyCall") will help here.

[songhanzhang]

Thanks for the suggestion. I have tried, but it remains not work.

julia> Pkg.build("PyCall")
Building Conda ─→ ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/915ebe6f0e7302693bdd8eac985797dba1d25662/build.log
Building PyCall → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/43d304ac6f0354755f1d60730ece8c499980f7ba/build.log

julia> using SymPy
[ Info: Precompiling SymPy [24249f21-da20-56a4-8eb1-6a02cf4ae2e6]
Python path configuration:
PYTHONHOME = '/Users/songhan.zhang/.julia/conda/3:/Users/songhan.zhang/.julia/conda/3'
PYTHONPATH = (not set)
program name = '/Users/songhan.zhang/.julia/conda/3/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.base_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.base_exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.platlibdir = 'lib'
sys.executable = '/Users/songhan.zhang/.julia/conda/3/bin/python'
sys.prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.exec_prefix = '/Users/songhan.zhang/.julia/conda/3'
sys.path = [
'/Users/songhan.zhang/.julia/conda/3/lib/python310.zip',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10',
'/Users/songhan.zhang/.julia/conda/3/lib/python3.10/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000000203001e00 (most recent call first):

ERROR: Failed to precompile SymPy [24249f21-da20-56a4-8eb1-6a02cf4ae2e6] to "/Users/songhan.zhang/.julia/compiled/v1.9/SymPy/jl_fCdvFZ".
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base ./loading.jl:2300
[3] compilecache
@ ./loading.jl:2167 [inlined]
[4] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1805
[5] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1660
[6] macro expansion
@ ./loading.jl:1648 [inlined]
[7] macro expansion
@ ./lock.jl:267 [inlined]
[8] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1611

[jverzani]

Hmm, let me try one more thing.

Someone just asked me about registering https://github.com/jverzani/SymPyCall.jl as installing sympy was problematic for them under PyCall, but not so under PythonCall.

You can install locally from github and see if that works for you. There are a few tests I need to fix that have bitrotted since I last looked at this, but the basic stuff is working just as expected.

[songhanzhang]

I think the issue is indeed related to the new version Julia. The package SymPy can be loaded properly as long as I re-install the Julia using an older version 1.6.7.

[jverzani]

This problematic. I'm in the middle of changing the backend to this package. Maybe one of those works:

SymPyPythonCall.jl might help, as the backend is PythonCall, not PyCall. People say Python package installation is easier.

SymPyPyCall.jl will replace SymPy but for now can be installed from GitHub