Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postprocessing fails when when a building simulation crashes at BSB #196

Closed
rajeee opened this issue Dec 15, 2020 · 0 comments · Fixed by #197
Closed

Postprocessing fails when when a building simulation crashes at BSB #196

rajeee opened this issue Dec 15, 2020 · 0 comments · Fixed by #197
Assignees
Labels
bug Something isn't working

Comments

@rajeee
Copy link
Contributor

rajeee commented Dec 15, 2020

Describe the bug
When a building simulation crashes in BSB, the run_building_d function here returns None, which results in dpouts having None. The results in one of the entries in results_jobxx.json being None. It results in a postprocessing crash here, since dpout will be none.

To Reproduce
Steps to reproduce the behavior:

  1. Have a simulation for a building crash at BSB. This is hard to simulate normally, but this particular incident happened due to I/O error here

Expected behavior
It shouldn't have caused the postprocessing to crash.

Logs

Please copy pertinent sections of log files here (e.g. sampling.out, job.out-*, postprocessing.out, sinularity-output.log, etc.)

####################
Traceback for building9040
Traceback (most recent call last):
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/eagle.py", line 299, in run_building_d
    return self.run_building(self.output_dir, self.cfg, i, upgrade_idx)
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/eagle.py", line 421, in run_building
    i
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/base.py", line 259, in cleanup_sim_dir
    f'{simout_ts_dir}/up{upgrade_id:02d}/bldg{building_id:07d}.parquet'
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/postprocessing.py", line 151, in write_dataframe_as_parquet
    parquet.write_table(tbl, f, flavor='spark')
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/pyarrow/parquet.py", line 1733, in write_table
    writer.write_table(table, row_group_size=row_group_size)
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/pyarrow/parquet.py", line 591, in write_table
    self.writer.write_table(table, row_group_size=row_group_size)
  File "pyarrow/_parquet.pyx", line 1433, in pyarrow._parquet.ParquetWriter.write_table
OSError: [Errno 5] Input/output error

Postprocessing crash:

####################
Traceback (most recent call last):
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/utils.py", line 61, in run_with_error_capture
    return func(*args, **kwargs)
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/eagle.py", line 747, in main
    batch.process_results()
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/base.py", line 779, in process_results
    postprocessing.combine_results(fs, self.results_dir, self.cfg, do_timeseries=do_timeseries)
  File "/shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/postprocessing.py", line 266, in combine_results
    dpout['job_id'] = job_id
TypeError: 'NoneType' object does not support item assignment

In file: /shared-projects/buildstock/envs/buildstock-0.19/lib/python3.7/site-packages/buildstockbatch/utils.py, module run_with_error_capture line: 61 
Local Variables: 'args':()
'kwargs':{}
'f':<_io.TextIOWrapper name='buildstockbatch_crash_details.log' mode='a' encoding='UTF-8'>
'text':'

Platform (please complete the following information):

  • Simulation platform: Eagle
  • BuildStockBatch version, branch, or sha: develop
  • OpenStudio-BuildStock version, branch, or sha: Irrelevant

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant