build system does not seem to respect CMAKE_INSTALL_PREFIX for Python API

[mfernan2]

When setting a custom install prefix, the Python API still tries to install to the system global directory:

$ mkdir build
$ cd build
$ cmake -DENABLE_PYTHON_INTERFACE=true -DCMAKE_INSTALL_PREFIX=/tmp/tmp.JOPT9jyHpq ..
...
$ make && make install
...
[100%] Linking CXX executable ../../stp_simple
[100%] Built target stp_simple-bin
[  1%] Built target ASTKind_header
[  3%] Built target util
[  3%] Built target stpglobals
[ 12%] Built target AST
[ 14%] Built target abstractionrefinement
[ 31%] Built target simplifier
[ 39%] Built target printer
[ 39%] Built target parser_headersmt2
[ 40%] Built target parser_headercvc
[ 41%] Built target parser_headersmt
[ 49%] Built target parser
[ 49%] Built target cppinterface
[ 50%] Built target cinterface
[ 84%] Built target abc
[ 85%] Built target constantbv
[ 87%] Built target stpmgr
[ 93%] Built target tosat
[ 94%] Built target sat
[ 95%] Built target stp
LEGACY: Copying c_interface.h to /tmp/tmp.gzxMDo2mop/stp/build/include/stp
LEGACY: Copying cpp_interface.h to /tmp/tmp.gzxMDo2mop/stp/build/include/stp
[ 95%] Built target CopyPublicHeaders
[ 97%] Built target stp-bin
[100%] Built target stp_simple-bin
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/cmake/STP/STPConfig.cmake
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/cmake/STP/STPConfigVersion.cmake
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/cmake/STP/STPTargets.cmake
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/cmake/STP/STPTargets-relwithdebinfo.cmake
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/libstp.so.2.3
-- Installing: /tmp/tmp.JOPT9jyHpq/lib/libstp.so
-- Set runtime path of "/tmp/tmp.JOPT9jyHpq/lib/libstp.so.2.3" to "lib"
-- Installing: /tmp/tmp.JOPT9jyHpq/include/stp/c_interface.h
-- Installing: /tmp/tmp.JOPT9jyHpq/bin/stp
-- Set runtime path of "/tmp/tmp.JOPT9jyHpq/bin/stp" to "lib"
-- Installing: /tmp/tmp.JOPT9jyHpq/bin/stp_simple
-- Set runtime path of "/tmp/tmp.JOPT9jyHpq/bin/stp_simple" to "lib"
CMake Error at bindings/python/stp/cmake_install.cmake:49 (file):
  file cannot create directory: /usr/lib/python2.7/dist-packages/stp.  Maybe
  need administrative privileges.
Call Stack (most recent call first):
  bindings/python/cmake_install.cmake:42 (include)
  bindings/cmake_install.cmake:42 (include)
  cmake_install.cmake:70 (include)


Makefile:117: recipe for target 'install' failed
make: *** [install] Error 1

Is this expected? I was assuming CMAKE_INSTALL_PREFIX would also redirect where Python modules are installed.

Config:

$ uname -rms
Linux 5.0.0-23-generic x86_64
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic
$ c++ --version
c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ python --version
Python 2.7.17
$ python2 --version
Python 2.7.17
$ python3 --version
Python 3.6.9

[rgov]

It seems wrong to me but using CMAKE_INSTALL_PREFIX also seems wrong. Do you know what the best practice is? Can we invoke Python and ask it where its site-packages are? Or would you rather we produce an .egg that you can install with pip?

[mfernan2]

Sorry, I'm not sure what best practice is for this. I suspect there isn't really one, because CMake assumes it's installing C/C++-compiled binaries/libs, not Python modules. E.g. GNUInstallDirs doesn't setup any Python package directories.

For my use case, we typically install everything related to a package under its own dedicated directory root. Then per-project setup scripts set/append PYTHONPATH to make any module dependencies available.

[msoos]

Hi,

I tend to agree with @rgov here -- it's actually not clear where/how python module installation is supposed to happen. In fact, we have tons-and-tons of issues with e.g. python being built with compiler X then we using compiler Z to build the plugin. But there is simply no standard way to get things like "what was used to compile python?" and that compiler may not be present anyway(!). So... this is a mess either way. setup.py, which is the supposedly standard way of doing this, barely helps with any of these. If you can find a way to fix this, that'd be nice -- but I would be very cautious of fixing it on my own, because it most likely will break it for others (using Mac, Windows, other Linux distros, etc.). Just my 2 cents,

Mate

[carlocab]

Ran into this while trying to package STP for Homebrew. In case anyone else is still looking, a fix/workaround seems to be to set PYTHON_LIB_INSTALL_DIR.

[aytey]

@carlocab after doing the install and the packages going into PYTHON_LIB_INSTALL_DIR, where is libstp.dylib? How does library_path.py look? Reasonable?

I'm not suggesting it is broken, I'm just checking that things are okay -- library_path.py is used for stp.py to find libstp.dylib, which then goes through ctypes. If library_types.py isn't completely correct, then stp.py won't be able to find libstp.dylib.

[carlocab]

libstp.dylib should be where you expect: ${CMAKE_INSTALL_PREFIX}/lib. I haven't had a look at library_path.py just yet. I'll update you when I have.

[aytey]

Ha, yeah, just because cmake puts it in ${CMAKE_INSTALL_PREFIX}/lib doesn't mean that's where library_path.py looks for it:

@CMAKE_LIBRARY_OUTPUT_DIRECTORY@/@LIBSTP_FILENAME@

Anyway, does brew install it to somewhere that's on DYLD_LIBRARY_PATH?

[carlocab]

Ok, had a look at library_path.py:

❯ tail -4 library_path.py
PATHS = [
    '/usr/local/Cellar/stp/2.3.3/lib/libstp.dylib',
    '/usr/local/Cellar/stp/2.3.3/bin/libstp.dylib'
]

Here, /usr/local/Cellar/stp/2.3.3 is what CMAKE_INSTALL_PREFIX was set to at build time, so that looks right. (Except for looking in the bin directory, but I guess that doesn't hurt.)

Anyway, does brew install it to somewhere that's on DYLD_LIBRARY_PATH?

It depends. In a default installation on an Intel Mac, things will get symlinked into /usr/local, which should be in DYLD_LIBRARY_PATH. Otherwise, no.