Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add github actions CI/CD for building cross-platform(Linux, MacOS and Windows) wheels for leveldb 1.23 and snappy 1.1.9 #129

Open
wants to merge 28 commits into
base: main
Choose a base branch
from

Conversation

liviaerxin
Copy link

@liviaerxin liviaerxin commented Jul 26, 2021

New Features:

  1. Support manylinux_x86_64, manylinux_aarch64, manylinux_i686 for cp37* cp38* cp39* in Linux using cibuildwheel
  2. Support x86_64 for cp37* cp38* cp39* and universal2 for cp39* in MacOS using cibuildwheel
  3. Support win32 and x64 for cp37* cp38* cp39* in Windows using cibuildwheel
  4. Support the version 1.23 of leveldb and the version 1.1.9 of snappy for all platforms(Linux, MacOS, Windows)
  5. Support pytest in CI

Check and verify my new CI result at github actions CI/CD

Released wheels at https://github.com/liviaerxin/plyvel/releases/tag/v1.3.0-CI

TODOs:

  1. Apply a new version such as 1.3.0 for plyvel.
  2. Modify function test_open_read_only_dir() in test/test_plyvel.py to pass pytest.
  3. Publish built wheels to pypi using github action.

@wbolster
Copy link
Owner

wow, i imagine this was quite some effort. thanks! 🙏

i will have a good look at this later.

@AustEcon
Copy link

AustEcon commented Jul 31, 2021

I did not realize this was here!!

I have been working on this myself on a private repo because I don't want the user to need to be required to do a system install of leveldb..

I have wheels for all platforms here for python 3.6 3.7 3.8 3.9 :
https://pypi.org/project/plyvel-wheels/#files

I've posted a PR for what it's worth. Whether you use it or not it's no worries 😄

@liviaerxin
Copy link
Author

Wow, @AustEcon thank you for presenting a idea to build windows wheels.

I will try to add it into github actions CI/CD later.

@liviaerxin
Copy link
Author

liviaerxin commented Aug 4, 2021

I have add to build windows wheel in github actions CI for leveldb 1.23 and snappy 1.1.9.

Please see new commits and CI results at plyvel github actions CI

@liviaerxin liviaerxin changed the title Add github actions CI/CD to build Linux and MacOS wheels for latest leveldb 1.23 and snappy 1.1.9 Add github actions CI/CD for building cross-platform(Linux, MacOS and Windows) wheels for leveldb 1.23 and snappy 1.1.9 Aug 5, 2021
@odidev
Copy link

odidev commented Feb 9, 2022

@wbolster, when are you planning to review this PR and merge it. It will be a great help to the machine users for which wheels are missing on PyPI.

@synodriver
Copy link

I've also made a fork to build wheels for windows which is build against the latest leveldb version(the main branch). Should we consider release the wheels derectly since install leveldb on windows is far more harder than that on unix? I could submit anothor pr for that.

@liviaerxin
Copy link
Author

I have released wheels built for Linux, Windows, and macOS. You can download your platform-specified plyvel wheel at https://github.com/liviaerxin/plyvel/releases/tag/v1.3.0-CI and install them directly without building both snappy and leveldb.

@wbolster wbolster removed their assignment Mar 9, 2022
@wbolster
Copy link
Owner

wbolster commented Mar 9, 2022

hey @liviaerxin wow, that is fantastic! i see you built on top of my build scripts (which i only ever ran successfully in a local docker environment, not in CI) 👍🏼

it would be great if we can get this cleaned up a bit and make it official, and trigger everything from this repo. we could even do automatic releases + pypi based on git tags etc. are you interested in helping getting that set up?

(i can also arrange write access to this repo of course)

@liviaerxin
Copy link
Author

hey @liviaerxin wow, that is fantastic! i see you built on top of my build scripts (which i only ever ran successfully in a local docker environment, not in CI) 👍🏼

it would be great if we can get this cleaned up a bit and make it official, and trigger everything from this repo. we could even do automatic releases + pypi based on git tags etc. are you interested in helping getting that set up?

(i can also arrange write access to this repo of course)

Hello @wbolster, I'm glad to participate in the following work and assist you to publish pypi/release automatically by CI to help people use plyvel package easily.

@liviaerxin
Copy link
Author

liviaerxin commented Mar 22, 2023

Hello @wbolster, I have not tracked your plyvel repo for a while and see it still does not provide binary wheels for cross-platform. Now I have some time and really wish your repo to be accessible for more people especially Windows users with you.

I can add some improvement to do automatic releases + pypi based on git tags, but it may need your helps such as setting some secret keys.

If you have other suggestions on CI, such as your mentioned code cleanup, feel free to tell me.

Then I can kick off the pull request again and simplify it by squashing commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants