Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat!: Pipeline YAML template using Airflow 2 operators #138

Merged
merged 18 commits into from Aug 11, 2021
Merged

feat!: Pipeline YAML template using Airflow 2 operators #138

merged 18 commits into from Aug 11, 2021

Commits on Aug 8, 2021

  1. new DAG operator imports for Airflow 2.0+

    @adlersantos
    adlersantos committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    edcbf3b View commit details
    Browse the repository at this point in the history

  2. support Airflow 2 operators when generating DAGs

    @adlersantos
    adlersantos committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    8482173 View commit details
    Browse the repository at this point in the history

  3. version compatibility checks for DAGs and Airflow environments

    @adlersantos
    adlersantos committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    3ec8449 View commit details
    Browse the repository at this point in the history

  4. tests to copy pipeline.yaml to dot folder (for airflow version spec)

    @adlersantos
    adlersantos committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    32b7ebc View commit details
    Browse the repository at this point in the history

  5. tests for version compatibility checks

    @adlersantos
    adlersantos committed Aug 8, 2021
    Configuration menu
    Copy the full SHA
    f780f90 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2021

  1. revised pipeline YAML template for Airflow 1 compat

    @adlersantos
    adlersantos committed Aug 9, 2021
    Configuration menu
    Copy the full SHA
    e4338f0 View commit details
    Browse the repository at this point in the history

  2. feat!: Upgrade dependencies to Airflow 2.1.1 (#135) 

    * revised dependendices for Airflow 2.1.1

* use proper variables file as required by Airflow 2

* changed README to use Airflow 2 requirement

* uncommented deploy_dag_versioning check

* feat: Upgrade `usa_names` pipeline to usse Airflow 2 operators and environment (#136)
    @adlersantos
    adlersantos committed Aug 9, 2021
    Configuration menu
    Copy the full SHA
    9249214 View commit details
    Browse the repository at this point in the history

  3. default to Airflow 2

    @adlersantos
    adlersantos committed Aug 9, 2021
    Configuration menu
    Copy the full SHA
    11bbb62 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2021

  1. README revisions for old and new YAML references

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    3621dd1 View commit details
    Browse the repository at this point in the history

  2. sample pipeline YAML template for Airflow 2 operators

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    6f68a2a View commit details
    Browse the repository at this point in the history

  3. add license header to unit-tests

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    2718430 View commit details
    Browse the repository at this point in the history

  4. require airflow_version to be explicitly stated

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    5f7a5ab View commit details
    Browse the repository at this point in the history

  5. set default Cloud Composer version to Airflow 2

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    f090654 View commit details
    Browse the repository at this point in the history

  6. tests to verify correct Airflow operators based on version specified

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    c37c2c8 View commit details
    Browse the repository at this point in the history

  7. additional GitHub check for unit testing Airflow 1 pipelines

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    ad52f11 View commit details
    Browse the repository at this point in the history

  8. set default Airflow version to 2

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    cc2a4af View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into airflow2-pipeline-yaml-sample

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    a1488d0 View commit details
    Browse the repository at this point in the history

  10. retrigger checks

    @adlersantos
    adlersantos committed Aug 10, 2021
    Configuration menu
    Copy the full SHA
    7c2706e View commit details
    Browse the repository at this point in the history