Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to avoid installing build dependency added by add_subdirectory? #852

Open
Char-Aznable opened this issue Jan 20, 2023 · 4 comments
Open

Comments

@Char-Aznable
Copy link

Hi, I am using scikit-build to build and install a project with pure c++, c++ pybind11 and pure python code. When I run pip install, I find that the third-party libraries added via cmake's fetchcontent or add_subdirectory required to build the c++ and c++ pybind11 code are installed into my python site-packages directory. For example, one of the dependency is a pure c++ header-only library that is only needed for building my python binding code but not needed for installation but somehow it's installed. Is there a way to avoid installing these third-party dependency? My setup.py looks like:

from skbuild import setup
from setuptools import find_packages

if __name__ == "__main__":
    setup(
        cmake_languages=("CXX", "CUDA"),
        packages=find_packages(
            where=".",
            exclude=["*mypkg1*"],
            ),
        package_dir={"mymodule": "mymodule"},
        exclude_package_data={"mymodule" : ["*_deps*"]},
        cmake_install_dir="mymodule",
    )
@Char-Aznable
Copy link
Author

Here is a minimal example. If I modify the example https://github.com/scikit-build/scikit-build-sample-projects/tree/master/projects/hello-pybind11 with this patch (basically just declaring and fetching a FetchContent dependency without eve n linking or using it):

diff --git a/projects/hello-pybind11/CMakeLists.txt b/projects/hello-pybind11/CMakeLists.txt
index d57d774..6699598 100644
--- a/projects/hello-pybind11/CMakeLists.txt
+++ b/projects/hello-pybind11/CMakeLists.txt
@@ -13,6 +13,16 @@ FetchContent_Declare(
 )
 FetchContent_MakeAvailable(pybind11)
 
+FetchContent_Declare(
+  tinyDNN 
+  GIT_REPOSITORY https://github.com/tiny-dnn/tiny-dnn.git
+  GIT_TAG        origin/master
+  GIT_SHALLOW 1
+  GIT_PROGRESS ON
+)
+
+FetchContent_MakeAvailable(tinyDNN)
+
 set(python_module_name _hello)
 pybind11_add_module(${python_module_name} MODULE
   src/hello/hello_py.cpp

this is enough to trigger the installation of the entire tiny-dnn repo, which is undesirable.

@Char-Aznable
Copy link
Author

Related to #500

@henryiii
Copy link
Contributor

Files (I'd really not call them dependencies) are controlled by setuptoools. So you'll want a MANIFEST.in and you can exclude/include things there.

In scikit-build-core, you'd instead have include/exclude fields in the pyproject.toml.

@Char-Aznable
Copy link
Author

Char-Aznable commented Jan 23, 2023

@henryiii Thanks for your response! The third-party libraries I mentioned are actually dependencies to build my project, as opposed to what is shown in the aforementioned example, which I deliberately avoid adding the build dependency to show the problem. In any case, the problem is the third-party projects have their own cmake install() commands, which by default will be triggered to install into the site-packages root, which is undesirable.

I then found the cmake_install_target option. It wasn't obvious from the documentation that "scikit-build will invoke cmake to install all add_subdirectory dependencies if not configured with cmake_install_target" is a thing to watch out for.

Anyway, thanks for this great tool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@henryiii @Char-Aznable and others