Remove residential default workflow generator #370

nmerket · 2023-05-19T16:29:49Z

Pull Request Description

Remove the residential default workflow generator and modifies tests to continue to work. Also some cleanup of other unnecessary files.

Checklist

Not all may apply

Code changes (must work)
Tests exercising your feature/bug fix (check coverage report on Checks -> BuildStockBatch Tests -> Artifacts)
Coverage has increased or at least not decreased. Update minimum_coverage in .github/workflows/ci.yml as necessary.
All other unit and integration tests passing
Update validation for project config yaml file changes
Update existing documentation
Run a small batch run on Eagle to make sure it all works if you made changes that will affect Eagle
Add to the changelog_dev.rst file and propose migration text in the pull request

github-actions · 2023-05-19T17:05:57Z

File	Coverage
All files	`83%`	✅
base.py	`88%`	✅
eagle.py	`75%`	✅
exc.py	`57%`	✅
local.py	`50%`	✅
postprocessing.py	`84%`	✅
utils.py	`96%`	✅
sampler/base.py	`79%`	✅
sampler/downselect.py	`33%`	✅
sampler/precomputed.py	`93%`	✅
sampler/residential_quota.py	`61%`	✅
test/test_docker.py	`33%`	✅
test/test_validation.py	`97%`	✅
workflow_generator/base.py	`90%`	✅
workflow_generator/commercial.py	`53%`	✅
workflow_generator/residential_hpxml.py	`76%`	✅

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against 282eeb4

joseph-robertson · 2023-05-22T16:00:02Z

docs/project_defn.rst

+- `ResStock National Baseline <https://github.com/NREL/resstock/blob/develop/project_national/national_baseline.yml>`_
+- `ResStock National with Upgrades <https://github.com/NREL/resstock/blob/develop/project_national/national_upgrades.yml>`_


nmerket · 2023-05-22T18:02:07Z

buildstockbatch/eagle.py

+    @classmethod
+    def validate_output_directory_eagle(cls, project_file):
+        cfg = get_project_configuration(project_file)
+        output_dir = path_rel_to_file(project_file, cfg['output_directory'])
+        if not (output_dir.startswith('/scratch') or output_dir.startswith('/projects')):
+            raise ValidationError(f"`output_directory` must be in /scratch or /projects, `output_directory` = {output_dir}")


Added this check because I accidentally set up a run to put my outputs in /home/nmerket. I should know better and made this mistake. This is a very common error for new people as well who aren't used to Eagle, so I made it so it won't let you try to save your outputs somewhere that doesn't make sense.

nmerket · 2023-05-22T18:04:48Z

I just successfully ran a small run on Eagle to make sure I didn't break anything and it worked.

nmerket · 2023-05-22T18:05:45Z

@joseph-robertson Could you take a quick look and approve if it looks good to you?

docs/changelog/changelog_dev.rst

joseph-robertson · 2023-05-22T22:38:07Z

docs/changelog/migration_0_21.rst

@@ -16,5 +16,4 @@ and documentation updates. See :doc:`changelog_0_21` for details.
 Schema Updates
 ==============

-Minor updates were made to the residential workflow generator. See
-:doc:`../workflow_generators/residential_default` for details.


Do we want to point to an older branch here?

Meh. Sounds like too much trouble.

joseph-robertson · 2023-05-22T22:38:35Z

docs/changelog/migration_2023_01_0.rst

@@ -22,7 +22,7 @@ Residential HPXML Workflow Changes
 Two new optional arguments were added to the Residential HPXML Workflow
 Generator: ``timeseries_num_decimal_places`` and
 ``include_timeseries_unmet_hours``. See
-:doc:`../workflow_generators/residential_default` for details about their usage.


Was this previously wrong?

Looks like it. The text above is for the HPXML workflow generator, so I changed the link.

joseph-robertson · 2023-05-22T22:40:13Z

buildstockbatch/test/test_inputs/enforce-validate-measures-bad-2.yml

+    build_existing_model:
+      simulation_control_timestep: 60
+      simulation_control_run_period_begin_month: 1
+      simulation_control_run_period_begin_day_of_month: 1.5 # intentional bad input data type for unit testing input validation
+      simulation_control_run_period_end_month: 12
+      simulation_control_run_period_end_day_of_month: 31
+      simulation_control_run_period_calendar_year: 2007
+
+    emissions:
+      - scenario_name: LRMER_MidCase_15
+        type: CO2e
+        elec_folder: data/cambium/LRMER_MidCase_15
+
+    utility_bills:
+      - scenario_name: Bills
+
+    simulation_output_report:
+      timeseries_frequency: huorly # intentionally misspelled for unit testing input validation
+      include_timeseries_total_consumptions: true
+      include_timeseries_fuel_consumptions: true
+      include_timeseries_end_use_consumptions: true
+      include_timeseries_emissions: true
+      include_timeseries_emission_fuels: true
+      include_timeseries_emission_end_uses: true
+      include_timeseries_hot_water_uses: true
+      include_timeseries_total_loads: true
+      include_timeseries_component_loads: true
+      include_timeseries_unmet_hours: true
+      include_timeseries_zone_temperatures: true
+      include_timeseries_airflows: true
+      include_timeseries_weather: true


Seems like you could probably (based on what you're testing) trim a lot of this as it may have no bearing on the actual test... (same comment for test ymls below)

True. I wasn't sure what that should be so I left it with what works. You're welcome to trim it down, or I'm happy to leave it as is.

spell validator correctly

joseph-robertson

Thanks for making all these updates/changes!

nmerket added 3 commits May 19, 2023 10:28

removing residential_default workflow generator

f8fac2a

removing unsed PyTest from setup.py

92f5aa8

removing QOI report from test reporting measures

7105ba0

nmerket added 2 commits May 19, 2023 11:06

removing docs reference to the example yaml files

f83b15b

adding to changelog_dev.rst [skip ci]

6331efd

joseph-robertson reviewed May 22, 2023

View reviewed changes

adding output directory validation to eagle.py

332a788

nmerket commented May 22, 2023

View reviewed changes

nmerket requested a review from joseph-robertson May 22, 2023 18:05

joseph-robertson reviewed May 22, 2023

View reviewed changes

Update docs/changelog/changelog_dev.rst

282eeb4

spell validator correctly

nmerket added this to the v2023.05.0 milestone May 23, 2023

joseph-robertson self-requested a review May 23, 2023 16:29

joseph-robertson approved these changes May 23, 2023

View reviewed changes

nmerket merged commit 25eb174 into develop May 23, 2023
6 checks passed

nmerket deleted the remove_res_default_workflow_generator branch May 23, 2023 22:56

nmerket mentioned this pull request May 25, 2023

Update sample yaml files and docs #358

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove residential default workflow generator #370

Remove residential default workflow generator #370

nmerket commented May 19, 2023 •

edited

github-actions bot commented May 19, 2023 •

edited

joseph-robertson May 22, 2023

nmerket May 22, 2023

nmerket commented May 22, 2023

nmerket commented May 22, 2023

joseph-robertson May 22, 2023

nmerket May 23, 2023

joseph-robertson May 22, 2023

nmerket May 23, 2023

joseph-robertson May 22, 2023

nmerket May 23, 2023

joseph-robertson left a comment

		- `ResStock National Baseline <https://github.com/NREL/resstock/blob/develop/project_national/national_baseline.yml>`_
		- `ResStock National with Upgrades <https://github.com/NREL/resstock/blob/develop/project_national/national_upgrades.yml>`_

Remove residential default workflow generator #370

Remove residential default workflow generator #370

Conversation

nmerket commented May 19, 2023 • edited

Pull Request Description

Checklist

github-actions bot commented May 19, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nmerket commented May 22, 2023

nmerket commented May 22, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joseph-robertson left a comment

Choose a reason for hiding this comment

nmerket commented May 19, 2023 •

edited

github-actions bot commented May 19, 2023 •

edited