Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated nightly release workflow #1499

Open
IvoDD opened this issue Apr 16, 2024 · 2 comments
Open

Automated nightly release workflow #1499

IvoDD opened this issue Apr 16, 2024 · 2 comments
Labels
enhancement New feature or request test engineer Issue that is good for an engineer in test to work on.

Comments

@IvoDD
Copy link
Collaborator

IvoDD commented Apr 16, 2024

Is your feature request related to a problem? Please describe.
Currently releases are painful because:

  1. We occasionally discover issues/bugs during or after a release which is bad
  2. The full iteration of doing a release is slow even if there are no issues (build times, conda forge prs etc.)
  3. We rarely do releases and power users need to wait a long time to get new features

Describe the solution you'd like
Having a separate automated nightly release process will have the following benefits:

  1. It will serve as a canary release and signal potential issues prior to a regular release (including issues with the release process itself).
  2. We can use the nightly build artifacts to quickly promote them to a full release.
  3. Power users will have access to new features sooner inside the nightly release.

This could look like an automated workflow which will run every night and push a new version to a separate python package e.g. arcticdb-nightly. Since arcticdb-nightly will not have a strong requirement for being bug-free the workflow can be fully automated without any human intervention.

We can then eventually transition internal Man Group use of arcticdb to the nightly build (and maybe allow for a flag to switch over to the latest regular release in case there are issues with the nightly build).

@IvoDD IvoDD added the enhancement New feature or request label Apr 16, 2024
@poodlewars
Copy link
Collaborator

We could probably do this by publishing a new RC every night or two, rather than a whole separate package.

@jjerphan
Copy link
Collaborator

Nightly releases definitely can help fixing bugs and regressions as soon as possible.

I also do not think a separate package is needed, a separate index or channel can be used generally so that one can install the nightly releases with:

python -m pip install --pre --upgrade --extra-index-url <channel_for_nightly_builds> arcticdb

Projects in the PyData / Scientific Python ecosystem have been discussing publishing nightly releases and eventually converged on an agreement and a set of tools to automate them which might be helpful for ArcticDB.

@poodlewars poodlewars added the test engineer Issue that is good for an engineer in test to work on. label Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request test engineer Issue that is good for an engineer in test to work on.
Projects
None yet
Development

No branches or pull requests

3 participants