Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] [PROPOSAL] use scikit-build for packaging #144

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

opadron
Copy link

@opadron opadron commented May 25, 2017

Hi, everyone.

I have a very preliminary fork where I was able to get wheels built for symengine using scikit-build. I wanted to share it with you all early, both to gauge interest in the idea of using scikit-build (and related projects) to assist with packaging and to invite feedback on our approach.

So far, I've only managed to get wheels built locally on my macbook, but I plan on fleshing out the proposed changes and making sure the result builds on windows, and linux, too.

@opadron
Copy link
Author

opadron commented May 25, 2017

cc @thewtex @jcfr @scikit-build/developers

@thewtex
Copy link

thewtex commented May 25, 2017

Very cool!

CC: @isuruf @certik

It should be possible to build wheels for Windows / macOS / Linux through AppVeyor / TravisCI / CircleCI.

@jcfr
Copy link
Contributor

jcfr commented May 25, 2017

Ditto. Well done.

It should be possible to build wheels for Windows / macOS / Linux through AppVeyor / TravisCI / CircleCI.

👍

else:
print("Value {} for USE_DISTUTILS treated as False".
format(use_distutils))
from skbuild import setup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try:
    from skbuild import setup
except ImportError:
    print('scikit-build is required to build from source.', file=sys.stderr)
    print('Please run:', file=sys.stderr)
    print('', file=sys.stderr)
    print('  python -m pip install scikit-build')
    sys.exit(1)

@isuruf
Copy link
Member

isuruf commented May 25, 2017

Nice. Thanks for this work.

I've evaluated scikit-build before, but didn't go with it because we'd be losing some features we have right now. From what I remember some of them are,

  1. Ability to build for multiple python versions and then install them. (Needed for creating deb packages in launchpad)
  2. Support for python setup.py build_ext -i (so that I can run the tests in the repo root)
  3. Support for python setup.py build --compiler= (I'm not sure whether this is possible already)

I'll have a look at scikit-build code to see how hard it is to add support for these.

@opadron
Copy link
Author

opadron commented May 25, 2017

@isuruf

1: I'm not sure if this is what you mean, but I don't think there's anything in principle stopping you from repeating the build with whatever versions of python you're willing to support. python setup.py install should continue to work just like with setuptools/distutils.

2: AFAIK, this would take a little work to add to skbuild. I'll need to check in on the latest concerning our support for in-place builds.

3: Selection and configuration of the compiler would be under the purview of CMake, so you could pass an option to CMake to select your compiler. We may have added --compiler as a convenience option to skbuild. @jcfr, do you recall either way?

@isuruf
Copy link
Member

isuruf commented May 27, 2017

From 1., I meant something like below.

python3.4 setup.py build
python3.5 setup.py build
python3.4 setup.py install
python3.5 setup.py install

Notice the difference between sk-build and setuptools. _skbuild/cmake-build vs build/lib.linux-x86_64-3.5

@opadron
Copy link
Author

opadron commented Sep 14, 2017

Hey, @isuruf sorry for the long silence, but I wanted to let you that we haven't forgotten about this investigation. I'm still very interested in exploring the possibility of improving scikit-build to meet the needs of your project, if you're willing to keep humoring us! :)

I now understand better what you were referring to on point 1. Separate build directories based on python version should be a relatively straight-forward add to skb. Do you think we should anticipate the desire to further distinguish environments based on other factors, like toolchain or target architecture?

On point 2, I had since been reminded by @jcfr that we do have some support for editable installs, through which I believe the binaries and other assets in a project may be built and assembled in-place (he would be the best person to talk to regarding the details). Would something like this address your needs?

@isuruf
Copy link
Member

isuruf commented Sep 14, 2017

Hey, @isuruf sorry for the long silence, but I wanted to let you that we haven't forgotten about this investigation. I'm still very interested in exploring the possibility of improving scikit-build to meet the needs of your project, if you're willing to keep humoring us! :)

I'm glad that you are still interested.

I now understand better what you were referring to on point 1. Separate build directories based on python version should be a relatively straight-forward add to skb. Do you think we should anticipate the desire to further distinguish environments based on other factors, like toolchain or target architecture?

We use distutils/setuptools info to generate the build directory.

def get_build_dir(dist):

On point 2, I had since been reminded by @jcfr that we do have some support for editable installs, through which I believe the binaries and other assets in a project may be built and assembled in-place (he would be the best person to talk to regarding the details). Would something like this address your needs?

I looked at it a while ago and if skbuild would set a cmake argument like -DBUILD_INPLACE=yes then I can copy the built extensions to the src dir as a post-install script so that it doesn't mess with the manifest that skbuild is using to figure out what to move to the site-packages directory.

@isuruf
Copy link
Member

isuruf commented Sep 14, 2017

FYI, we are building wheels separately here, https://github.com/symengine/symengine-wheels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants