Compiling PyGIMLi from Source on Apple M-Based Macs

[robinthibaut]

Hello everyone,

I've encountered some difficulties with the instructions given for compiling PyGIMLi from source on Macs. They were a great starting point, but I had to work through some roadblocks to make it work. I'd like to share adjusted instructions that should provide a good starting point for others in the community. While some steps may need tweaking for your individual systems, this guide should give you a fair idea:

1. Install Homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

2. Install essential libraries:

brew install boost 
brew install cmake 
brew install git 
brew install wget
brew install mercurial
brew install xorgproto

3. Install Miniforge: Download the latest version from Conda-forge's Github page.

curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh

4. Prepare a Python environment with an M-optimized version of Python through this Miniforge Conda, ensure to install necessary python packages:

conda create -n pygimli_env python=3.9
conda activate pygimli_env
conda install -c conda-forge boost numpy scipy matplotlib

5. Clone the PyGIMLi repository, create a build directory, and build PyGIMLi:

mkdir -p ~/src/gimli
cd ~/src/gimli
git clone https://github.com/gimli-org/gimli.git
cd gimli/
mkdir build
cd build/

Importantly, the paths listed in the CMake command need to correspond to the version of Python being used, which you can check using which python3 and python3 --version

PYTHON_EXEC=$(which python3)
PYTHON_INC=$(python3 -c 'import sysconfig; print(sysconfig.get_path("include"))')
PYTHON_LIB=$(python3-config --configdir)

export CPLUS_INCLUDE_PATH=$PYTHON_INC

cmake .. -DPYTHON_EXECUTABLE=$PYTHON_EXEC -DPYTHON_LIBRARY=$PYTHON_LIB -DPYTHON_INCLUDE_DIR=$PYTHON_INC 
make -j 8
make pygimli

You can test your installation with:

python3 -c 'import pygimli as pg; print(pg.__version__)'

Watch out: You might notice that PyGIMLi creates an .so file named lib_pygimli_.so, make sure to rename it to _pygimli_.so since _pygimli_.so is the name PyGIMLi internally uses.

I hope this guide can be of some use.

Cheers,
Robin

[halbmy]

Dear Robin,
thank you so much for sharing your findings with the community! Maybe there are (except @florian-wagner who generates the conda packages) others around to test and share their experience. Would be interesting to found out whether this depends on the OS version, kernel, chip etc.

Good to have the Mac users on board. Are the conda versions working for you?

[robinthibaut]

Dear Thomas,

I attempted to install pygimli using conda on my M2 Mac as per the instructions, but I encountered a couple of issues when running conda create -n pg -c gimli -c conda-forge pygimli=1.4.3.

First, there was a problem loading the conda-libmamba-solver plugin due to a missing library, specifically libarchive.13.dylib.

Second, I had trouble finding the tetgen package in the channels gimli and conda-forge. From the error messages, it seems tetgen might not be available for my specific architecture osx-arm64.

That's why I decided to compile it from source - it now works smoothly!
It would be nice for the community to have access to conda versions working out of the box though. I'd be happy to help test different setups or solutions for Macs if anyone needs that.

Note that I also have access to an Intel-based Mac, on which the conda installation succeeded.

[carsten-forty2]

Hello Robin,

Watch out: You might notice that PyGIMLi creates an .so file named lib_pygimli_.so, make sure to rename it to pygimli.so since pygimli.so is the name PyGIMLi internally uses.

ahh .. I found the same weird leading 'lib' also for the windows build, which probably comes from a recent cmake version. I recently added a special cmake target to rename it for the windows build and should add this target for the mac build to.

Thanks a lot for the building instructions .. we will add a link to this thread into the documentation, or can we copy it directly?

Cheers,
Carsten

[robinthibaut]

Hello Carsten,
Yes, I felt really bummed out after getting the error no module named pgcore, although compilation apparently ran successfully. Luckily, after some digging, I found out about the .so file. Please use/copy/edit my instructions as you wish!
Cheers,
Robin

[carsten-forty2]

Bad me to try blaming cmake for my own faults. I messed up the cmake script and accidently removed the lib prefix. Its fixed now and the library output name is correct again for all platforms.

[ruboerner]

Hello all,

many thanks to @robinthibaut for sharing the installation steps for Mac OS!

I could successfully compile pygimli after some modifications to the guide given above :

1. As umfpack was missing in my homebrew installation, I had to add suite-sparse to the list of essential libraries:

homebrew install suite-sparse

2. Further, cmake struggled with finding lapack and openblas, so I had to modify the build commands (as per item 5 in the list above):

PYTHON_EXEC=$(which python3)
PYTHON_INC=$(python3 -c 'import sysconfig; print(sysconfig.get_path("include"))')
PYTHON_LIB=$(python3-config --configdir)

export CPLUS_INCLUDE_PATH=$PYTHON_INC

export CMAKE_PREFIX="/opt/homebrew/opt/lapack;/opt/homebrew/opt/openblas"

cmake .. -DPYTHON_EXECUTABLE=$PYTHON_EXEC -DPYTHON_LIBRARY=$PYTHON_LIB -DPYTHON_INCLUDE_DIR=$PYTHON_INC -DCMAKE_PREFIX_PATH=$CMAKE_PREFIX
make -j 8
make pygimli

After a cd .. I got the following result:

python -c 'import pygimli as pg; print(pg.__version__)'
1.4.6+43.gc2601998 (master)

I hope this addition might help.

Cheers,
Ralph

[florian-wagner]

Thanks for the feedback. We need to update the compilation instructions for Mac OS on the website.

[florian-wagner]

Dear @robinthibaut @ruboerner,

could you please try if conda create -n pg15 -c gimli -c conda-forge pygimli=1.5.0 works on your Mac machines?

Thanks and best wishes
Florian

[robinthibaut]

Dear @florian-wagner,
The installation went smoothly, and I successfully ran an ERT inversion. Thanks!!
Robin

[florian-wagner]

Thanks Robin! Your feedback (and compilation instructions above) are much appreciated! If you have pyGIMLi compiled manually or if you simply install the pgcore package and use the git-sources for pyGIMLi as described here: https://www.pygimli.org/contrib.html#d-contribute-to-code You are very welcome to contribute a PR if you spot anything in the code or have developed something useful. We appreciate every little contribution (from typos over new examples to new functionality).

Stay in touch
Florian

[ruboerner]

Hi @florian-wagner,

I can also confirm that the installation works. It runs smoothly!

Thanks,
Ralph