2.8.2.3

What's Changed

• fix: 1951 - acoustic suppression list date format by @lelilia in #1952
• feat(DENG-2863): add data-observability-dev to the list of projects covered by artifact_deployment by @kik-kik in #1954
• fix: 1952 Acoustic Suppression List DENG 3082 date format by @lelilia in #1955
• feat: Create shredder backfill dag by @BenWu in #1941
• fix: Remove dev requirements from Docker image and improve Makefile by @sean-rose in #1953
• feat(managed-backfills): DENG-983 Add DAGs for initiating and completing backfills by @ANich in #1956
• feat(managed-backfills): DENG-989 - Use overwatch Slack app for managed backfills by @ANich in #1957
• fix(dags): mad_server secrets are managed on GSM. by @edugfilho in #1958
• fix(managed-backfills): typo in Slack message, add owners by @ANich in #1959
• feat: Default reattach_on_restart parameter to True in bigquery_etl_query function by @akkomar in #1961

Full Changelog: 2.8.2.2...2.8.2.3

2.8.2.2

What's Changed

• feat(): add acoustic supression list export by @kik-kik in #1951

Full Changelog: 2.8.2.1...2.8.2.2

2.8.2.1

What's Changed

• chore: Add PR template. by @mikaeld in #1943
• Revert "Bring back SQL generation in artifact deployments" by @scholtzan in #1940
• Update mad_server owner by @akkomar in #1945
• chore: Replace gke_command usage with GKEPodOperator. by @mikaeld in #1948
• feat: Add repo tag by @lelilia in #1947
• feat: Add DAG for running opmon backfills by @scholtzan in #1946
• feat: Make args parameter optional in operational_monitoring_backfill by @scholtzan in #1949
• feat: Upgrade Python to 3.11.8 and remove trailing whitespaces by @kik-kik in #1942

Full Changelog: 2.8.2.0...2.8.2.1

Airflow 2.8.2.0

What's Changed

• fix(): Remove FivetranSensor to avoid double wait for sync to complete by @kik-kik in #1922
• Limit Fivetran task concurrency so it only tries to run one sync per connector at a time by @sean-rose in #1923
• Fix bqetl_artifact_deployment.publish_views to publish views in alternate projects by @sean-rose in #1925
• Publish glam managed views by @edugfilho in #1926
• chore: Cleanup dependencies. by @mikaeld in #1918
• DSRE-1519 Fix secrets in merino_jobs dag by @quiiver in #1927
• DSRE-1520 Manage secrets for adm_export dag in GSM by @quiiver in #1928
• bqetl_artifact_deployment max_active_runs and remove SQL generation by @scholtzan in #1930
• Bring back SQL generation in artifact deployments by @scholtzan in #1931
• Change Fivetran Airflow operators to run synchronously so the task concurrency limit is respected by @sean-rose in #1932
• Bug 1821773 - Update debugging notes for probe scraper and bqetl deploy by @BenWu in #1933
• fix(dockerfile): google-cloud-sdk renamed google-cloud-cli. by @mikaeld in #1938
• chore(dags): Delete overwatch.py by @mikaeld in #1937
• feat: Upgrade Airflow from 2.7.3 to 2.8.2 by @sean-rose in #1935

Full Changelog: 2.7.3.1...2.8.2.0

Airflow 2.7.3.1

What's Changed

• [DISCO-2699] Added xcom pull for diff file access by @taddes in #1865
• Change bqetl_backfill to no_triage by @fbertsch in #1867
• Move glean usage queries to bigquery-etl by @scholtzan in #1858
• mozaggregator uses main_v5 by @edugfilho in #1869
• Revert "mozaggregator uses main_v5" by @edugfilho in #1870
• feat(DENG-2066): deprecating internet_outages.global_outages_v1 by @kik-kik in #1868
• feat(DENG-2064): deprecating DIM checks for rolling_cohorts as they are no longer needed by @kik-kik in #1871
• feat(DENG-2065): deprecating DIM checks for cohorts_daily_statistics as they are no longer needed by @kik-kik in #1872
• Update socorro_import.py overview comment by @willkg in #1875
• feat(DENG-2063): deprecating DIM checks for telemetry_derived.active_users_aggregates_v1 as migrated to ETL checks by @kik-kik in #1873
• feat(DENG-2072): deprecating DIM checks for telemetry_derived.unified_metrics_v1 by @kik-kik in #1877
• feat(DENG-2067): deprecating DIM checks for org_mozilla_fenix_nightly.baseline_clients_last_seen by @kik-kik in #1874
• feat(DENG-2070): data_monitoring DAG removed, all checks in this DAG have been migrated to ETL checks by @kik-kik in #1878
• feat(DENG-1990): deprecated dim checks for active_users_aggregates mobile by @kik-kik in #1880
• fix(): Removing MySQL repo from our Airflow docker image sources due to incorrect GPG Key error by @kik-kik in #1885
• feat(DENG-1990): deprecated dim checks for active_users_aggregates firefox_desktop by @kik-kik in #1881
• feat(DENG-1990): deprecated active_users_aggregates dim checks DAG (mobile + desktop) by @kik-kik in #1884
• Remove temporary fix by @relud in #1888
• Remove dthorn as owner of dags and tasks by @relud in #1883
• Fixes #1889 - Add a dag definition to schedule broken-site-report-ml job by @ksy36 in #1890
• FXCTO-13 Update dap_collector's output destination by @akkomar in #1887
• Add new DAG ga4_site_metrics_summary_backfill by @kwindau in #1891
• Fix parameter in new DAG ga4_site_metrics_summary_backfill by @kwindau in #1892
• Run query initialization before schema deploys by @scholtzan in #1893
• Remove unnecessary no_partition parameter by @ANich in #1894
• Have bqetl_artifact_deployment publish new glean_usage-generated tables by @sean-rose in #1896
• DSRE-1522, DSRE-52: Remove mozaggregator_prerelease DAG from Airflow by @edugfilho in #1899
• chore: Delete create_gke_cluster. by @mikaeld in #1904
• chore: Delete GKENatPodOperator. by @mikaeld in #1903
• Remove GLAM dev DAG by @edugfilho in #1900
• Remove adjust_import and android_client_history_sim dags by @fbertsch in #1879
• GLAM move secrets outside of Airflow by @edugfilho in #1906
• chore: delete burnham and bq_sensor operator. by @mikaeld in #1905
• chore: delete code leftovers. by @mikaeld in #1909
• Remove bugzilla DAG by @scholtzan in #1910
• chore(plugins): Remove unused plugins and dependencies. by @mikaeld in #1911
• Add project moz-fx-data-glam-prod-fca7 to the DAG bqetl_artifact_depl… by @lucia-vargas-a in #1916
• Remove stray && in bqetl_artifact_deployment by @scholtzan in #1919
• Issue 4135: Added publish metadata task by @wwyc in #1912
• DSRE-1526 - Move probe_scraper secrets to GSM by @akkomar in #1917
• DSRE-1526 - Move probe_scraper secret to GSM by @akkomar in #1921
• feat(DENG-2682): Update fivetran imports to use airflow-provider-fivetran-async Airflow Fivetran provider by @kik-kik in #1914

New Contributors

• @kwindau made their first contribution in #1891
• @wwyc made their first contribution in #1912

Full Changelog: 2.7.3.0...2.7.3.1

Airflow 2.7.3.0

What's Changed

• Add link to GKE cluster in top bar menu by @akkomar in #1840
• Bqetl backfill 4real by @fbertsch in #1841
• Don't deploy glean_usage tables with artifacts by @fbertsch in #1843
• Enable checks for first_seen and daily tables by @fbertsch in #1842
• DSRE-1456 - Archive main_v5 partitions after 37 months by @relud in #1845
• Fix table name syntax for bq cli by @relud in #1847
• feat(): Updating DIM related scheduling to match that of aggregates DAG by @kik-kik in #1846
• bug(1854406): updating python_moztelemetry package to use the new git tag reference by @kik-kik in #1849
• docs(): Updated graphics telemetry DAG docs by @kik-kik in #1850
• DENG-1946. Update dependency of KPI dashboard to bqetl_analytics_aggr… by @lucia-vargas-a in #1852
• Add DAG to backfill 3 days of ga_sessions each day by @fbertsch in #1855
• Change glam owner by @edugfilho in #1857
• Add dummy task to test bhr gcs bucket data transfer by @alekhyamoz in #1856
• Changing the cluster for bhr gcs test job by @alekhyamoz in #1859
• Remove the test bhr gcs task by @alekhyamoz in #1860
• Change the bucket to gcp for bhr data and remove sync job by @alekhyamoz in #1862
• feat: Upgrade to Airflow 2.7.3 by @mikaeld in #1854

New Contributors

• @lucia-vargas-a made their first contribution in #1852

Full Changelog: 2.6.3.3...2.7.3.0

Airflow 2.6.3.3

What's Changed

• glam_glean_import tasks reattach to pods on restart by @edugfilho in #1837
• Add link to GKE cluster in top bar menu by @akkomar in #1838

Full Changelog: 2.6.3.2...2.6.3.3

Airflow 2.6.3.2

What's Changed

• DENG-1024: firefox_public_data_report exports to GCS instead of S3 by @edugfilho in #1775
• DENG-1024: firefox_public_data_report exports to GCS instead of S3 (prod) by @edugfilho in #1798
• This should not work. However... by @chelseatroy in #1799
• [DISCO-2601] update ownership of merino_jobs to disco-team by @taddes in #1796
• Remove python command in favor of... by @chelseatroy in #1800
• Deliberately use the wrong docker image… by @chelseatroy in #1801
• fix: remove .env file by @mikaeld in #1806
• Put "echo" and "HELLO WORLD" in separate entries in command list by @chelseatroy in #1802
• Correct docker image, wrong command by @chelseatroy in #1808
• Ctroy stdv echo by @chelseatroy in #1809
• Ctroy stdv echo by @chelseatroy in #1810
• Ctroy stdv echo by @chelseatroy in #1811
• Point to the real file by @chelseatroy in #1813
• Switch aggregates jobs to use workaround branch and image by @whd in #1814
• Add to triage notes for probe_scraper DAG by @mikewilli in #1815
• DO-1283 - Add note on deprecating mobile_aggregates by @akkomar in #1817
• Update external task markers in copy_deduplicate by @scholtzan in #1818
• Set reattach_on_restart=True in copy_deduplicate by @akkomar in #1820
• Deduplicate main_use_counter and _remainder outside of copy_deduplicate_all by @akkomar in #1822
• Add note on current status of mozaggregator_prerelease by @akkomar in #1819
• Fixes #1823 - Add dag definition to schedule webcompat-kb job by @ksy36 in #1824
• Bug 1852630 - Rename main_remainder_v4 to main_v5 by @relud in #1825
• Add jsilverman's email to the Airflow Job by @bochocki in #1826
• add triage/confidential as valid tag by @mikewilli in #1827
• DENG-1023 crash sym stores results on gcs instead of s3 by @edugfilho in #1829
• "DENG 1022 - Update bhr_collection DAG to write to GCS instead…" by @alekhyamoz in #1830
• Revert ""DENG 1022 - Update bhr_collection DAG to write to GCS instead…"" by @alekhyamoz in #1832
• DENG-476 - Update shredder tasks for main_v5 migration by @relud in #1831
• Rename KPI forecasting directory by @bochocki in #1833
• Glam - reattach pods on restart by @edugfilho in #1834
• Update path to agree with docker-etl changes by @bochocki in #1835
• feat: Move utils, operators and glam_subdags out of dags directory by @mikaeld in #1807

New Contributors

• @ksy36 made their first contribution in #1824

Full Changelog: 2.6.3.1...2.6.3.2

Airflow 2.6.3.2 RC2

2.6.3.2rc2

linter

Airflow 2.6.3.2 RC1

2.6.3.2rc1

update requirements.txt