Python tests do not work without AudioTK Python libraries installed

[SpotlightKid]

The Python tests do not work without the Python libraries installed, so the test can not be run from the source or build tree without installing first, which is a problem e.g. for distribution packagers.

The reason is that they import the ATK modules in the form from ATK.Xyz import xyz, but the directory for the ATK python package is only created on installation of the modules. In the build tree there is no ATK directory and hence no ATK package. So even adding the build/Python directory to PYTHONPATH for the tests wouldn't work.

I don't see a quick solution for this. IMHO either the build process should be altered in such a way that the modules are created in a proper package directory structure under build/Python/ATK or a testing environment should be used, that creates a temporary installation of the Python package (like e.g. tox.)

[SpotlightKid]

This ticket arose out of of issue #15.

[mbrucher]

Thanks a lot!
A preliminary hack that I added on the current SIMD branch is DISABLE_PYTHON_TESTS as I had lots of issues, even after make install, on appveyor and travis-ci.

But it's a workaround only, I need to figure out a proper way to fix this. The refactoring to streamline the Python generation should help, as I can also generate the init.py file from them and then just rename the parent folder accordingly.

[mbrucher]

Hey,
Can you test the develop branch for this? I moved all Python modules to an ATK subfolder, and the build folder should also contain the required init.py files. I can't really test for now, as this doesn't work for OS X or Windows (as the libraries are under a Debug/Release subfolder).

[SpotlightKid]

When I run make test after compilation (you can see the command used to compile here), I get this output:

Running tests...
Test project /home/chris/src/arch/aur/audiotk-git/src/audiotk/build
      Start  1: Adaptive
 1/13 Test  #1: Adaptive .........................   Passed    0.51 sec
      Start  2: Core
 2/13 Test  #2: Core .............................   Passed   36.16 sec
      Start  3: Delay
 3/13 Test  #3: Delay ............................   Passed    9.84 sec
      Start  4: Distortion
 4/13 Test  #4: Distortion .......................   Passed    0.04 sec
      Start  5: Dynamic
 5/13 Test  #5: Dynamic ..........................   Passed    1.13 sec
      Start  6: EQ
 6/13 Test  #6: EQ ...............................   Passed    3.48 sec
      Start  7: IO
 7/13 Test  #7: IO ...............................   Passed    0.04 sec
      Start  8: Mock
 8/13 Test  #8: Mock .............................   Passed    0.06 sec
      Start  9: Preamplifier
 9/13 Test  #9: Preamplifier .....................   Passed    0.05 sec
      Start 10: Reverberation
10/13 Test #10: Reverberation ....................   Passed    0.73 sec
      Start 11: Special
11/13 Test #11: Special ..........................   Passed   19.77 sec
      Start 12: Tools
12/13 Test #12: Tools ............................   Passed    0.61 sec
      Start 13: Utility
13/13 Test #13: Utility ..........................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 13

I'm not sure whether this did in fact run the Python tests.

[mbrucher]

Definitely didn't run the Python tests (they are prefixed with Py).
You should see make behaving like https://travis-ci.org/mbrucher/AudioTK/jobs/334227656
-- Found PythonInterp: /opt/python/2.7.14/bin/python (found version "2.7.14")
-- Found PythonLibs: /opt/python/2.7.14/lib/libpython2.7.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.3.dev0
-- Found NumPy: /opt/python/2.7.14/lib/python2.7/site-packages/numpy/core/include (found version "1.14.0")
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
If there isn't, then it means Python was not successfully found.

[SpotlightKid]

I guess the output -- Found PythonInterp: comes from line 2 in tests/Python/CMakeLists.txt. What are the pre-requisites that this finds my system Python installation?

[mbrucher]

It comes from here actually: https://github.com/mbrucher/AudioTK/blob/master/CMakeLists.txt#L188
I think usually the Python install is retrieved automatically (it's handled by pybind11). For non standard install path, I use these (travis-ci): -DPYTHON_LIBRARY=/opt/python/${Python_ADDITIONAL_VERSIONS}/lib/libpython${PYTHON_VERSION}.so -DPYTHON_INCLUDE_DIR=/opt/python/${Python_ADDITIONAL_VERSIONS}/include/python${PYTHON_VERSION} -DPYTHON_EXECUTABLE=/opt/python/${Python_ADDITIONAL_VERSIONS}/bin/python

[SpotlightKid]

Still getting only 13 tests. This is my configuration / compilation sequence (in an up-to-date git checkout):

  mkdir build
  git checkout develop
  git pull
  git submodule init
  git submodule update
  cd build
  local py_ver=$(python -c 'import sys; print("%s.%s" % sys.version_info[:2])')
  cmake .. \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DDISABLE_EIGEN_WARNINGS=1 \
    -DENABLE_PYTHON=1 \
    -DPYTHON_LIBRARY=/usr/lib/libpython${py_ver}m.so \
    -DPYTHON_INCLUDE_DIR=/usr/include/python${py_ver}m \
    -DPYTHON_EXECUTABLE=/usr/bin/python${py_ver} \
    -DENABLE_TESTS=1 \
    -DBUILD_DOC=1 \
    -DENABLE_GPL=1
  make
  make test

The output from cmake seems to indicate that the Python installation is found:

-- The C compiler identification is GNU 7.2.1
-- The CXX compiler identification is GNU 7.2.1
-- Check for working C compiler: /usr/lib/ccache/bin/cc
-- Check for working C compiler: /usr/lib/ccache/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/lib/ccache/bin/c++
-- Check for working CXX compiler: /usr/lib/ccache/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
--  Build SIMD: OFF
--  Build shared libraries: ON
--  Build static libraries: OFF
--  Build tests: 1
-- Performing Test COMPILER_SUPPORTS_CXX17
-- Performing Test COMPILER_SUPPORTS_CXX17 - Success
-- Performing Test COMPILER_SUPPORTS_PTHREAD
-- Performing Test COMPILER_SUPPORTS_PTHREAD - Success
-- Performing Test COMPILER_SUPPORTS_ftree_vectorize
-- Performing Test COMPILER_SUPPORTS_ftree_vectorize - Success
-- Performing Test COMPILER_HAS_SUGGEST_OVERRIDE
-- Performing Test COMPILER_HAS_SUGGEST_OVERRIDE - Success
-- Performing Test COMPILER_SUPPORTS_DEPRECATED_DECLARATIONS
-- Performing Test COMPILER_SUPPORTS_DEPRECATED_DECLARATIONS - Success
-- Performing Test COMPILER_SUPPORTS_IGNORED_ATTRIBUTES
-- Performing Test COMPILER_SUPPORTS_IGNORED_ATTRIBUTES - Success
-- Found Git: /usr/bin/git  
-- Found LIBSNDFILE: /usr/include  
-- Boost version: 1.66.0
-- Found FFTW: /usr/lib/libfftw3.so  
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   unit_test_framework
--   system
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   unit_test_framework
--   system
-- Found PythonInterp: /usr/bin/python3.6 (found version "3.6.4") 
-- Found PythonLibs: /usr/lib/libpython3.6m.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.3.dev0
-- Found NumPy: /usr/lib/python3.6/site-packages/numpy/core/include (found version "1.14.0") 
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.14") found components:  doxygen dot 
Release build.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/chris/src/arch/aur/audiotk-git/src/audiotk/build

I'm out of my depth here with cmake and ctest. I'd need more specific instructions if I'm to test this further.

BTW, the compilation output is incredibly noisy. Is there a way to reduce the number of warnings, especially the following:

• "member X of class Y is not documented"
• "the name X supplied as the second argument in the \file statement matches the following input files"

[mbrucher]

I wonder if there is some remnant configuration somewhere...
For the noisy output, that's the documentation output. Nothing I can do except add more doc in some places (but for private variables, I don't document them, nor the overriden methods).
Can you try by adding -DDISABLE_PYTHON_TESTS=OFF? Seems like it's on for a strange reason. I don't understand how ctest wouldn't see them except if this is on (this happens in tests/Python/CMakeLists.txt if((ENABLE_PYTHON STREQUAL "ON") AND (DISABLE_PYTHON_TESTS STREQUAL "OFF"))).

[SpotlightKid]

Can you try by adding -DDISABLE_PYTHON_TESTS=OFF?

Same results.

I wonder if there is some remnant configuration somewhere...

The build process I use for AUR packages always starts with a fresh clone of of an up-to-date git checkout.

[mbrucher]

Seems like everything should work. That's odd. I'll have to check a few things.

[mbrucher]

I tried it on an embedded Linux and the latest develop version, it worked as expected, the tests did have all the Python scripts :/
So I have no clue why it doesn't work on your platform. You may want to add a few message(STATUS "${entry}") inside the python/ATK/CMakeLists.txt file to display which files are really parsed.