New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Onboard EPA historical air quality dataset #221
Conversation
…riable value for each.
…rate as intended in AF
…nting some pipelines to complete successfully.
…ml; Fixed flake8 issue in csv_transform
@adlersantos Ready for review. Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good in general. Just a few comments.
datasets/epa_historical_air_quality/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/_images/run_csv_transform_kub/requirements.txt
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/annual_summaries/pipeline.yaml
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/co_daily_summary/pipeline.yaml
Outdated
Show resolved
Hide resolved
datasets/epa_historical_air_quality/co_hourly_summary/pipeline.yaml
Outdated
Show resolved
Hide resolved
@adlersantos I have done research into trying to understand why this PR keeps failing at the Run Tests point. Is there some recent change to the code behind these checks? What changes do I need to make to ensure that these checks pass? Thanks! |
@adlersantos I dont see any button in github UI that allows me to rerun the checks without making code changes, so I had to close and reopen the PR in order to trigger the tests. If you know a better way to re-trigger the checks, or if closing and re-opening the PR is acceptable then please let me know. |
@adlersantos Please review this code at your earliest convenience. Thanks! |
@adlersantos I have made the changes. Please review. Thanks. |
…atch size in order to resolve memory pressure resulting in DAG failure.
@adlersantos All requested changes are completed. Please review again. Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot!
Description
Included:
Annual summaries
CO Daily Summary
CO Hourly Summary
HAP Daily Summary
HAP Hourly Summary
Lead Daily Summary
NO2 Daily Summary
NO2 Hourly Summary
NONOxNOy Daily Summary
NONOxNOy Hourly Summary
Ozone Daily Summary
Ozone Hourly Summary
PM 10 Daily Summary
PM10 Hourly Summary
PM25 Frm Hourly Summary
PM25 NonFrm Daily Summary
PM25 NonFrm Hourly Summary
PM25 Speciation Daily Summary
PM25 Speciation Hourly Summary
Pressure Daily Summary
Pressure Hourly Summary
RH and DP Daily Summary
RH and DP Hourly Summary
SO2 Daily Summary
SO2 Hourly Summary
Temperature Daily Summary
Temperature Hourly Summary
VOC Daily Summary
VOC Hourly Summary
Wind Daily Summary
Wind Hourly Summary
Checklist
Note: Delete items below that aren't applicable to your pull request.
datasets/epa_historical_air_quality
and nothing outside of that directory.