When using default TMY weather URL receiving error-- OSError: [Errno 28] No space left on device

[whiphi92]

Describe the bug
Receiving immediate failure during the sampling step of buildstockbatch when pointing to a URL for weather files.

e.g., weather_files_url: https://data.nrel.gov/system/files/156/BuildStock_TMY3_FIPS.zip

To Reproduce
You can reproduce that by using buildstock-2023.06.0 environment and running national_baseline.yml on Eagle while pointing to the weather file URL.

Logs

sampling.out:

DEBUG:2023-08-29 08:40:27:buildstockbatch.base:Using OpenStudio version: 3.6.1 with SHA: bb9481519e
DEBUG:2023-08-29 08:40:27:__main__:Output directory = /scratch/pwhite/durham_buildstock_v2
DEBUG:2023-08-29 08:40:27:buildstockbatch.base:Downloading weather files
Traceback (most recent call last):
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/base.py", line 115, in _get_weather_files
    f.write(chunk)
OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 875, in <module>
    main()
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/utils.py", line 107, in run_with_error_capture
    return func(*args, **kwargs)
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 840, in main
    batch = EagleBatch(args.project_filename)
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 80, in __init__
    weather_dir = self.weather_dir  # noqa E841
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/eagle.py", line 168, in weather_dir
    self._get_weather_files()
  File "/shared-projects/buildstock/envs/buildstock-2023.06.0/lib/python3.10/site-packages/buildstockbatch/base.py", line 112, in _get_weather_files
    with tempfile.TemporaryFile() as f:
OSError: [Errno 28] No space left on device

real	0m13.196s
user	0m2.673s
sys	0m2.517s

Platform (please complete the following information):

• Simulation platform: Eagle
• BuildStockBatch version, branch, or sha: buildstock-2023.06.0

[rajeee]

Seems like either you have run out of space in /scratch/pwhite/ or there is some problem in eagle. You can try to create a file manually in /scratch/pwhite/ see what happens.

[whiphi92]

I troubleshooted this with @nmerket and he requested I post this issue. We do not think it is a space issue.

[nmerket]

This happens when downloading and extracting weather files to a temporary file. I think on Linux that lands in /tmp. On Eagle compute nodes that's a RAM disk with very little capacity. I need to confirm that, though. If that's the case it's an easy fix, just point it to /tmp/scratch, which is a 1 TB attached hard drive. The weird thing is that this hasn't happened before, which is kind of surprising.