Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get airflow version from importlib.metadata rather than hard-coding #12786

Merged
merged 3 commits into from Dec 4, 2020
Merged

Get airflow version from importlib.metadata rather than hard-coding #12786

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f9fccbc
Get airflow version from importlib.metadata rather than hard-coding
ashb Dec 3, 2020
ae150c4
fixup! Get airflow version from importlib.metadata rather than hard-c…
ashb Dec 3, 2020
dbea186
fixup! Get airflow version from importlib.metadata rather than hard-c…
ashb Dec 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 0 additions & 5 deletions .pre-commit-config.yaml
View file
Open in desktop
Expand Up @@ -440,11 +440,6 @@ repos:
files: ^airflow/www/.*\.(css|scss|sass)$
# Keep dependency versions in sync w/ airflow/www/package.json
additional_dependencies: ['stylelint@13.3.1', 'stylelint-config-standard@20.0.0']
- id: version-sync
name: Version sync
files: ^airflow/version.py$|setup.py
entry: ./scripts/ci/pre_commit/pre_commit_sync_version.sh
language: system
- id: providers-init-file
name: Provider init file
pass_filenames: false
Expand Down
1 change: 0 additions & 1 deletion Dockerfile.ci
View file
Open in desktop
Expand Up @@ -306,7 +306,6 @@ RUN yarn --cwd airflow/www install --frozen-lockfile --no-cache
COPY setup.py ${AIRFLOW_SOURCES}/setup.py
COPY setup.cfg ${AIRFLOW_SOURCES}/setup.cfg

COPY airflow/version.py ${AIRFLOW_SOURCES}/airflow/version.py
COPY airflow/__init__.py ${AIRFLOW_SOURCES}/airflow/__init__.py

ARG UPGRADE_TO_LATEST_CONSTRAINTS="false"
Expand Down
2 changes: 0 additions & 2 deletions STATIC_CODE_CHECKS.rst
View file
Open in desktop
Expand Up @@ -174,8 +174,6 @@ require Breeze Docker images to be installed locally:
----------------------------------- ---------------------------------------------------------------- ------------
``update-extras`` Updates extras in the documentation.
----------------------------------- ---------------------------------------------------------------- ------------
``version-sync`` Synchronizes versions setup.py <-> airflow/version.py.
----------------------------------- ---------------------------------------------------------------- ------------
``yamllint`` Checks yaml files with yamllint.
=================================== ================================================================ ============

Expand Down
11 changes: 10 additions & 1 deletion airflow/version.py
View file
Open in desktop
Expand Up @@ -17,4 +17,13 @@
# under the License.
#

version = '2.0.0b3'
__all__ = ['version']

try:
from importlib import metadata
except ImportError:
import importlib_metadata as metadata

version = metadata.version('apache-airflow')

del metadata
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This line isn't strictly needed, it was just so that the only symbol in this module was version. For no god reason really)

2 changes: 0 additions & 2 deletions dev/README.md
View file
Open in desktop
Expand Up @@ -200,8 +200,6 @@ pip install twine
- Set proper permissions for the pypirc file:
`$ chmod 600 ~/.pypirc`

- Confirm that `airflow/version.py` is set properly.


## Hardware used to prepare and verify the packages

Expand Down
6 changes: 3 additions & 3 deletions dev/README_RELEASE_AIRFLOW.md
View file
Open in desktop
Expand Up @@ -65,7 +65,7 @@ The Release Candidate artifacts we vote upon should be the exact ones we vote ag
export AIRFLOW_REPO_ROOT=$(pwd)
```

- Set your version to 1.10.2 in `airflow/version.py` (without the RC tag)
- Set your version to 1.10.2 in `setup.py` (without the RC tag)
- Commit the version change.

- Tag your release
Expand Down Expand Up @@ -147,7 +147,7 @@ svn commit -m "Add artifacts for Airflow ${VERSION}"
At this point we have the artefact that we vote on, but as a convenience to developers we also want to
publish "snapshots" of the RC builds to pypi for installing via pip. To do this we need to

- Edit the `airflow/version.py` to include the RC suffix.
- Edit the `setup.py` to include the RC suffix.

- Build the package:

Expand Down Expand Up @@ -176,7 +176,7 @@ https://test.pypi.org/project/apache-airflow/#files
- Again, confirm that the package is available here:
https://pypi.python.org/pypi/apache-airflow

- Throw away the change - we don't want to commit this: `git checkout airflow/version.py`
- Throw away the change - we don't want to commit this: `git checkout setup.py`

It is important to stress that this snapshot should not be named "release", and it
is not supposed to be used by and advertised to the end-users who do not read the devlist.
Expand Down
1 change: 0 additions & 1 deletion dev/provider_packages/enter_breeze_provider_package_tests.sh
View file
Open in desktop
Expand Up @@ -26,7 +26,6 @@ function enter_breeze_with_mapped_sources() {
-v "${AIRFLOW_SOURCES}/setup.py:/airflow_sources/setup.py:cached" \
-v "${AIRFLOW_SOURCES}/setup.cfg:/airflow_sources/setup.cfg:cached" \
-v "${AIRFLOW_SOURCES}/airflow/__init__.py:/airflow_sources/airflow/__init__.py:cached" \
-v "${AIRFLOW_SOURCES}/airflow/version.py:/airflow_sources/airflow/version.py:cached" \
-v "${AIRFLOW_SOURCES}/empty:/opt/airflow/airflow:cached" \
-v "${AIRFLOW_SOURCES}/scripts/in_container:/opt/airflow/scripts/in_container:cached" \
-v "${AIRFLOW_SOURCES}/dev/import_all_classes.py:/opt/airflow/dev/import_all_classes.py:cached" \
Expand Down
5 changes: 2 additions & 3 deletions scripts/ci/libraries/_initialization.sh
View file
Open in desktop
Expand Up @@ -134,8 +134,8 @@ function initialization::initialize_base_variables() {
# otherwise it will use files/airflow-breeze-config/init.sh
export INIT_SCRIPT_FILE=${INIT_SCRIPT_FILE:=""}

# Read airflow version from the version.py
AIRFLOW_VERSION=$(grep version "${AIRFLOW_SOURCES}/airflow/version.py" | awk '{print $3}' | sed "s/['+]//g")
# Read airflow version from the setup.py.
AIRFLOW_VERSION=$(python "${AIRFLOW_SOURCES}/setup.py" --version)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer not to run Python here if possible. While this is a reasonable expectation that Python is installed in the hos, there are still some cases where people will have python2 by default for example and i would rather keep it bash only. There is another case in the future where we will use those scripts in code spaces environment and vs code to run docker-only environment and i would not like to have to depend on Python there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scripts/ci/libraries/_initialization.sh: line 138: python: command not found

Failed anyway :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course that all untill we rewrite breeze in python - which might change the whole approach: #12282

export AIRFLOW_VERSION

# Whether credentials should be forwarded to inside the docker container
Expand Down Expand Up @@ -194,7 +194,6 @@ function initialization::initialize_files_for_rebuild_check() {
"setup.cfg"
"Dockerfile.ci"
".dockerignore"
"airflow/version.py"
"airflow/www/package.json"
"airflow/www/yarn.lock"
"airflow/www/webpack.config.js"
Expand Down
1 change: 0 additions & 1 deletion scripts/ci/libraries/_md5sum.sh
View file
Open in desktop
Expand Up @@ -102,7 +102,6 @@ function md5sum::calculate_md5sum_for_all_files() {
# * setup.py
# * setup.cfg
# * Dockerfile.ci
# * airflow/version.py
#
# This is needed because we want to skip rebuilding of the image when only airflow sources change but
# Trigger rebuild in case we need to change dependencies (setup.py, setup.cfg, change version of Airflow
Expand Down
29 changes: 0 additions & 29 deletions scripts/ci/pre_commit/pre_commit_sync_version.sh
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion scripts/ci/provider_packages/ci_install_and_test_provider_packages.sh
View file
Open in desktop
Expand Up @@ -44,7 +44,6 @@ function run_test_package_import_all_classes() {
-v "${AIRFLOW_SOURCES}/setup.py:/airflow_sources/setup.py:cached" \
-v "${AIRFLOW_SOURCES}/setup.cfg:/airflow_sources/setup.cfg:cached" \
-v "${AIRFLOW_SOURCES}/airflow/__init__.py:/airflow_sources/airflow/__init__.py:cached" \
-v "${AIRFLOW_SOURCES}/airflow/version.py:/airflow_sources/airflow/version.py:cached" \
-v "${AIRFLOW_SOURCES}/empty:/opt/airflow/airflow:cached" \
-v "${AIRFLOW_SOURCES}/scripts/in_container:/opt/airflow/scripts/in_container:cached" \
-v "${AIRFLOW_SOURCES}/dev/import_all_classes.py:/opt/airflow/dev/import_all_classes.py:cached" \
Expand Down
1 change: 0 additions & 1 deletion setup.py
View file
Open in desktop
Expand Up @@ -30,7 +30,6 @@

logger = logging.getLogger(__name__)

# This is automatically maintained in sync via pre-commit from airflow/version.py
version = '2.0.0b3'

PY3 = sys.version_info[0] == 3
Expand Down