Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Column Mismatch in Post-processing #265

Closed
aspeake opened this issue Feb 4, 2022 · 0 comments · Fixed by #266
Closed

Column Mismatch in Post-processing #265

aspeake opened this issue Feb 4, 2022 · 0 comments · Fixed by #266
Assignees
Labels
bug Something isn't working

Comments

@aspeake
Copy link
Contributor

aspeake commented Feb 4, 2022

Describe the bug
On the restructure-v3 branch, the post-processing hits an error in the combine_results() method: ValueError: Metadata mismatch found in from_delayed. The error occurs when calling: df = dask.compute(results_df_groups.get_group(upgrade_id))[0]

To Reproduce
Running post-processing with the residential_hpxml wokflow generator
yml: https://github.com/NREL/resstock/blob/run/restructure-v3/project_national/resstock_hpxml_2018.yml

Logs

Postprocessing.out traceback:

Traceback (most recent call last):
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/aspeake/buildstock/buildstockbatch-2/buildstockbatch/eagle.py", line 741, in <module>
    main()
  File "/home/aspeake/buildstock/buildstockbatch-2/buildstockbatch/utils.py", line 98, in run_with_error_capture
    return func(*args, **kwargs)
  File "/home/aspeake/buildstock/buildstockbatch-2/buildstockbatch/eagle.py", line 731, in main
    batch.process_results()
  File "/home/aspeake/buildstock/buildstockbatch-2/buildstockbatch/base.py", line 572, in process_results
    postprocessing.combine_results(fs, self.results_dir, self.cfg, do_timeseries=do_timeseries)
  File "/home/aspeake/buildstock/buildstockbatch-2/buildstockbatch/postprocessing.py", line 315, in combine_results
    df = dask.compute(results_df_groups.get_group(upgrade_id))[0]
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/dask/base.py", line 570, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/client.py", line 2722, in get
    results = self.gather(packed, asynchronous=asynchronous, direct=direct)
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/client.py", line 1983, in gather
    asynchronous=asynchronous,
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/client.py", line 866, in sync
    self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/utils.py", line 327, in sync
    raise exc.with_traceback(tb)
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/utils.py", line 310, in f
    result[0] = yield future
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/tornado/gen.py", line 762, in run
    value = future.result()
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/distributed/client.py", line 1842, in _gather
    raise exception.with_traceback(traceback)
  File "/shared-projects/buildstock/envs/dev_aspeake_2/lib/python3.7/site-packages/dask/dataframe/utils.py", line 408, in check_meta
    "%s" % ((" in `%s`" % funcname if funcname else ""), errmsg)
ValueError: Metadata mismatch found in `from_delayed`.

Partition type: `pandas.core.frame.DataFrame`
+------------------------------------------------------------------+---------+----------+
| Column                                                           | Found   | Expected |
+------------------------------------------------------------------+---------+----------+
| 'ApplyUpgrade.applicable'                                        | float64 | object   |
| 'ApplyUpgrade.upgrade_name'                                      | float64 | object   |
| 'BuildExistingModel.applicable'                                  | object  | bool     |
| 'QOIReport.applicable'                                           | object  | bool     |
| 'ReportSimulationOutput.add_timeseries_dst_column'               | object  | bool     |
| 'ReportSimulationOutput.add_timeseries_utc_column'               | object  | bool     |
| 'ReportSimulationOutput.applicable'                              | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_airflows'             | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_component_loads'      | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_emissions'            | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_end_use_consumptions' | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_fuel_consumptions'    | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_hot_water_uses'       | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_total_loads'          | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_weather'              | object  | bool     |
| 'ReportSimulationOutput.include_timeseries_zone_temperatures'    | object  | bool     |
| 'UpgradeCosts.applicable'                                        | object  | bool     |
+------------------------------------------------------------------+---------+----------+

real	2m19.599s
user	0m1.540s
sys	0m1.258s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants