-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ResStock-HPXML #208
Merged
Merged
ResStock-HPXML #208
Changes from all commits
Commits
Show all changes
97 commits
Select commit
Hold shift + click to select a range
b101461
stubbing out ResidentialHpxmlWorkflowGenerator
nmerket babd75a
Update residential hpxml workflow generator and test.
joseph-robertson 38d30e0
Update docs with hpxml workflow generator page.
joseph-robertson 3ad5c48
Include measures at resources/hpxml-measures.
joseph-robertson 30e7c74
Fix some formatting.
joseph-robertson c4c22b6
Few more minor docs changes.
joseph-robertson 5f574c8
Get os 3.1 image and correct timeseries file.
joseph-robertson 2f0228c
Some build existing model args have changed.
joseph-robertson 60c3fe8
Remove more unit_ from build existing model.
joseph-robertson 5b3d901
Clean up a bunch of stuff.
joseph-robertson 9f6d31d
Mount hpxml measures folder.
joseph-robertson 458929e
Only mount hpxml folder if it exists.
joseph-robertson baea180
Typo.
joseph-robertson d462e79
Change mount dir in docker and aws.
joseph-robertson 89cea4f
Revert hpxml folder changes and try mount separately instead.
joseph-robertson 36e6931
Try fixing resources bind mount.
joseph-robertson a9dd5ea
Fix how do_timeseries is determined.
joseph-robertson cd52db7
Get upgrade costs from datapoint out json.
joseph-robertson 1054eb2
Add run options to hpxml workflow.
joseph-robertson c18dc16
Report upgrade costs in results csv.
joseph-robertson 83bb566
pinning to numpy<1.20
nmerket 9295171
updating numpy>=1.20 and pyarrow>=3.0 as those seem to work with each…
nmerket 8561012
Merge branch 'develop' into restructure-v3
joseph-robertson 45687c2
Update for new schedules csv filename.
joseph-robertson 78bd3b4
Run option fast, and call server directory cleanup.
joseph-robertson a22012a
Switch from data_point_out json to results json.
joseph-robertson febc51b
Get started_at and completed_at from job files.
joseph-robertson e8fe426
Clean up default args and add optional debug arg.
joseph-robertson 6c14585
Merge pull request #217 from NREL/restructure-v3-runoptions
joseph-robertson e6a8bf8
Get residential hpxml test passing.
joseph-robertson 0b3e65f
Merge branch 'develop' into restructure-v3
joseph-robertson 729be65
Clean up started_at and completed_at stamps.
joseph-robertson c239132
Add measures and reporting_measures into hpxml generator.
joseph-robertson 201fff3
Update the hpxml generator docs.
joseph-robertson dffd251
Merge remote-tracking branch 'origin/develop' into restructure-v3
nmerket 74e4199
Merge remote-tracking branch 'origin/develop' into restructure-v3
nmerket 477213d
Determine invalid dps using results json.
joseph-robertson 04eaf08
Remove debug line.
joseph-robertson fd84328
Minor fixes in postprocessing.
joseph-robertson 660b1ab
Update changelog_dev.
joseph-robertson 73e465c
Merge branch 'develop' into restructure-v3
joseph-robertson 8696089
Update simg to os320.
joseph-robertson e3e25dc
Merge branch 'develop' into restructure-v3
joseph-robertson f0168ed
Update version and sha to os 3.2.1.
joseph-robertson eb906bc
Merge branch 'develop' into restructure-v3
joseph-robertson c324327
Merge branch 'develop' into restructure-v3
joseph-robertson a8d4f71
Update hpxml workflow generator with server dir cleanup changes.
joseph-robertson e266a89
Add to schema yml.
joseph-robertson b05a159
Reporting measure name changes.
joseph-robertson b1acb96
Update hpxml workflow test.
joseph-robertson 5481750
Format.
joseph-robertson b288f84
Change more sim output report names.
joseph-robertson 739af63
Even more measure name changes.
joseph-robertson 74f1302
Produce hourly end use and total loads timeseries by default.
joseph-robertson 91a7427
Update residential hpxml workflow generator docs.
joseph-robertson 89678ab
Fix default os version.
joseph-robertson 7115c51
Change timeseries default behavior back to none.
joseph-robertson 0da0cd4
Bump to official os v330.
joseph-robertson 75ff695
Merge branch 'develop' into restructure-v3
joseph-robertson cf83525
update check for timeseries csv request
aspeake f526107
Merge branch 'restructure-v3' of github.com:NREL/buildstockbatch into…
aspeake e6a1aa3
correct yml argument name for simulation output report
aspeake 276265f
Bump to official os v330.
joseph-robertson b049197
Update workflow generator for co2 emissions timeseries.
joseph-robertson dd3379f
Another generator update.
joseph-robertson 8ce7bf3
Allow argument list in yml for co2_emissions.
joseph-robertson 3fb4086
Update arg names.
joseph-robertson 465ca13
Continue to update workflow generator for emissions arguments.
joseph-robertson dac9681
Move yml emissions arg up a level.
joseph-robertson a8f4296
Updates to workflow generator docs.
joseph-robertson 26da8e8
Update workflow generator docs.
joseph-robertson 340b3d0
Merge branch 'restructure-v3' into restructure-v3-cambium
joseph-robertson fc5689d
Clean up.
joseph-robertson 3a47168
Updates to schema.
joseph-robertson c10dff2
Include emissions fuel args.
joseph-robertson d181d08
Update validate method.
joseph-robertson 8b7233e
Relax requires on fuel args.
joseph-robertson 0d5ef86
Revert and try float.
joseph-robertson 782f6e2
The validator is num not float.
joseph-robertson e377939
Convert to str.
joseph-robertson 6764522
Add dst and utc time columns by default.
joseph-robertson 84e3a09
Merge branch 'restructure-v3' into restructure-v3-cambium
joseph-robertson f94bd6f
Merge branch 'develop' into restructure-v3
joseph-robertson 7e9f7e4
Remove optionals from workflow generator.
joseph-robertson 27f9ceb
Merge branch 'restructure-v3' into restructure-v3-cambium
joseph-robertson 6b2b116
Update workflow docs.
joseph-robertson 8d90bcf
More docs updates.
joseph-robertson fec2f3d
Merge pull request #259 from NREL/restructure-v3-cambium
joseph-robertson 4a0eeb4
remove overwrite of TimeDST column
aspeake 346eb32
Merge pull request #264 from NREL/restructure-v3-dst-fix
aspeake 4cb4380
Merge branch 'develop' into restructure-v3
joseph-robertson 174279c
Fixes for backward compatibility.
joseph-robertson f5f5c11
Fix str.
joseph-robertson 24f53c7
Merge branch 'github_actions' into restructure-v3
joseph-robertson 28051fc
Fix.
joseph-robertson 5d7670f
Update for removing fast run option.
joseph-robertson 5b6296a
Merge branch 'develop' into restructure-v3
joseph-robertson File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
223 changes: 223 additions & 0 deletions
223
buildstockbatch/workflow_generator/residential_hpxml.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,223 @@ | ||
# -*- coding: utf-8 -*- | ||
|
||
""" | ||
buildstockbatch.workflow_generator.residential_hpxml | ||
~~~~~~~~~~~~~~~ | ||
This object contains the residential classes for generating OSW files from individual samples | ||
|
||
:author: Joe Robertson | ||
:copyright: (c) 2021 by The Alliance for Sustainable Energy | ||
:license: BSD-3 | ||
""" | ||
|
||
import datetime as dt | ||
import json | ||
import logging | ||
import re | ||
import yamale | ||
|
||
from .base import WorkflowGeneratorBase | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
||
class ResidentialHpxmlWorkflowGenerator(WorkflowGeneratorBase): | ||
|
||
@classmethod | ||
def validate(cls, cfg): | ||
"""Validate arguments | ||
|
||
:param cfg: project configuration | ||
:type cfg: dict | ||
""" | ||
schema_yml = """ | ||
measures_to_ignore: list(str(), required=False) | ||
build_existing_model: map(required=False) | ||
emissions: list(include('scenario-spec'), required=False) | ||
reporting_measures: list(include('measure-spec'), required=False) | ||
simulation_output_report: map(required=False) | ||
server_directory_cleanup: map(required=False) | ||
--- | ||
scenario-spec: | ||
scenario_name: str(required=True) | ||
type: str(required=True) | ||
elec_folder: str(required=True) | ||
gas_value: num(required=False) | ||
propane_value: num(required=False) | ||
oil_value: num(required=False) | ||
wood_value: num(required=False) | ||
measure-spec: | ||
measure_dir_name: str(required=True) | ||
arguments: map(required=False) | ||
""" | ||
workflow_generator_args = cfg['workflow_generator']['args'] | ||
schema_yml = re.sub(r'^ {8}', '', schema_yml, flags=re.MULTILINE) | ||
schema = yamale.make_schema(content=schema_yml, parser='ruamel') | ||
data = yamale.make_data(content=json.dumps(workflow_generator_args), parser='ruamel') | ||
yamale.validate(schema, data, strict=True) | ||
return True | ||
|
||
def reporting_measures(self): | ||
"""Return a list of reporting measures to include in outputs""" | ||
workflow_args = self.cfg['workflow_generator'].get('args', {}) | ||
return [x['measure_dir_name'] for x in workflow_args.get('reporting_measures', [])] | ||
|
||
def create_osw(self, sim_id, building_id, upgrade_idx): | ||
""" | ||
Generate and return the osw as a python dict | ||
|
||
:param sim_id: simulation id, looks like 'bldg0000001up01' | ||
:param building_id: integer building id to use from the sampled buildstock.csv | ||
:param upgrade_idx: integer index of the upgrade scenario to apply, None if baseline | ||
""" | ||
# Default argument values | ||
workflow_args = { | ||
'build_existing_model': {}, | ||
'measures': [], | ||
'simulation_output_report': {}, | ||
'server_directory_cleanup': {} | ||
} | ||
workflow_args.update(self.cfg['workflow_generator'].get('args', {})) | ||
|
||
logger.debug('Generating OSW, sim_id={}'.format(sim_id)) | ||
|
||
sim_ctl_args = {} | ||
|
||
bld_exist_model_args = { | ||
'building_id': building_id, | ||
'sample_weight': self.n_datapoints / self.cfg['baseline']['n_buildings_represented'] | ||
} | ||
if 'measures_to_ignore' in workflow_args: | ||
bld_exist_model_args['measures_to_ignore'] = '|'.join(workflow_args['measures_to_ignore']) | ||
bld_exist_model_args.update(sim_ctl_args) | ||
bld_exist_model_args.update(workflow_args['build_existing_model']) | ||
|
||
if 'emissions' in workflow_args: | ||
emissions = workflow_args['emissions'] | ||
emissions_map = [['emissions_scenario_names', 'scenario_name'], | ||
['emissions_types', 'type'], | ||
['emissions_electricity_folders', 'elec_folder'], | ||
['emissions_natural_gas_values', 'gas_value'], | ||
['emissions_propane_values', 'propane_value'], | ||
['emissions_fuel_oil_values', 'oil_value'], | ||
['emissions_wood_values', 'wood_value']] | ||
for arg, item in emissions_map: | ||
bld_exist_model_args[arg] = ','.join([str(s.get(item)) for s in emissions]) | ||
|
||
sim_out_rep_args = { | ||
'timeseries_frequency': 'none', | ||
'include_timeseries_fuel_consumptions': False, | ||
'include_timeseries_end_use_consumptions': True, | ||
'include_timeseries_emissions': False, | ||
'include_timeseries_hot_water_uses': False, | ||
'include_timeseries_total_loads': True, | ||
'include_timeseries_component_loads': False, | ||
'include_timeseries_zone_temperatures': False, | ||
'include_timeseries_airflows': False, | ||
'include_timeseries_weather': False, | ||
'add_timeseries_dst_column': True, | ||
'add_timeseries_utc_column': True | ||
} | ||
sim_out_rep_args.update(workflow_args['simulation_output_report']) | ||
|
||
osw = { | ||
'id': sim_id, | ||
'steps': [ | ||
{ | ||
'measure_dir_name': 'BuildExistingModel', | ||
'arguments': bld_exist_model_args | ||
} | ||
], | ||
'created_at': dt.datetime.now().isoformat(), | ||
'measure_paths': [ | ||
'measures', | ||
'resources/hpxml-measures' | ||
], | ||
'run_options': { | ||
'skip_zip_results': True | ||
} | ||
} | ||
|
||
osw['steps'].extend(workflow_args['measures']) | ||
|
||
server_dir_cleanup_args = { | ||
'retain_in_osm': False, | ||
'retain_in_idf': True, | ||
'retain_pre_process_idf': False, | ||
'retain_eplusout_audit': False, | ||
'retain_eplusout_bnd': False, | ||
'retain_eplusout_eio': False, | ||
'retain_eplusout_end': False, | ||
'retain_eplusout_err': False, | ||
'retain_eplusout_eso': False, | ||
'retain_eplusout_mdd': False, | ||
'retain_eplusout_mtd': False, | ||
'retain_eplusout_rdd': False, | ||
'retain_eplusout_shd': False, | ||
'retain_eplusout_sql': False, | ||
'retain_eplustbl_htm': False, | ||
'retain_sqlite_err': False, | ||
'retain_stdout_energyplus': False, | ||
'retain_stdout_expandobject': False, | ||
'retain_schedules_csv': True | ||
} | ||
server_dir_cleanup_args.update(workflow_args['server_directory_cleanup']) | ||
|
||
osw['steps'].extend([ | ||
{ | ||
'measure_dir_name': 'ReportSimulationOutput', | ||
'arguments': sim_out_rep_args | ||
}, | ||
{ | ||
'measure_dir_name': 'ReportHPXMLOutput', | ||
'arguments': {} | ||
}, | ||
{ | ||
'measure_dir_name': 'UpgradeCosts', | ||
'arguments': {} | ||
}, | ||
{ | ||
'measure_dir_name': 'ServerDirectoryCleanup', | ||
'arguments': server_dir_cleanup_args | ||
} | ||
]) | ||
|
||
if upgrade_idx is not None: | ||
measure_d = self.cfg['upgrades'][upgrade_idx] | ||
apply_upgrade_measure = { | ||
'measure_dir_name': 'ApplyUpgrade', | ||
'arguments': { | ||
'run_measure': 1 | ||
} | ||
} | ||
if 'upgrade_name' in measure_d: | ||
apply_upgrade_measure['arguments']['upgrade_name'] = measure_d['upgrade_name'] | ||
for opt_num, option in enumerate(measure_d['options'], 1): | ||
apply_upgrade_measure['arguments']['option_{}'.format(opt_num)] = option['option'] | ||
if 'lifetime' in option: | ||
apply_upgrade_measure['arguments']['option_{}_lifetime'.format(opt_num)] = option['lifetime'] | ||
if 'apply_logic' in option: | ||
apply_upgrade_measure['arguments']['option_{}_apply_logic'.format(opt_num)] = \ | ||
self.make_apply_logic_arg(option['apply_logic']) | ||
for cost_num, cost in enumerate(option.get('costs', []), 1): | ||
for arg in ('value', 'multiplier'): | ||
if arg not in cost: | ||
continue | ||
apply_upgrade_measure['arguments']['option_{}_cost_{}_{}'.format(opt_num, cost_num, arg)] = \ | ||
cost[arg] | ||
if 'package_apply_logic' in measure_d: | ||
apply_upgrade_measure['arguments']['package_apply_logic'] = \ | ||
self.make_apply_logic_arg(measure_d['package_apply_logic']) | ||
|
||
build_existing_model_idx = \ | ||
[x['measure_dir_name'] == 'BuildExistingModel' for x in osw['steps']].index(True) | ||
osw['steps'].insert(build_existing_model_idx + 1, apply_upgrade_measure) | ||
|
||
if 'reporting_measures' in workflow_args: | ||
for reporting_measure in workflow_args['reporting_measures']: | ||
if 'arguments' not in reporting_measure: | ||
reporting_measure['arguments'] = {} | ||
reporting_measure['measure_type'] = 'ReportingMeasure' | ||
osw['steps'].insert(-1, reporting_measure) # right before ServerDirectoryCleanup | ||
|
||
return osw |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍