Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle empty data_point_out.json #426

Merged
merged 4 commits into from Jan 2, 2024
Merged

handle empty data_point_out.json #426

merged 4 commits into from Jan 2, 2024

Conversation

rajeee
Copy link
Contributor

@rajeee rajeee commented Jan 2, 2024

Pull Request Description

Sometimes, data_point_out.json can be present, but empty. This leads to obscure error downstream (see screenshot). This PR fixes the issue by treating empty json as not present.

MicrosoftTeams-image (2)

Checklist

Not all may apply

  • Code changes (must work)
  • Tests exercising your feature/bug fix (check coverage report on Checks -> BuildStockBatch Tests -> Artifacts)
  • Coverage has increased or at least not decreased. Update minimum_coverage in .github/workflows/coverage.yml as necessary.
  • All other unit and integration tests passing
  • Update validation for project config yaml file changes
  • Update existing documentation
  • Run a small batch run on Kestrel/Eagle to make sure it all works if you made changes that will affect Kestrel/Eagle
  • Add to the changelog_dev.rst file and propose migration text in the pull request

@rajeee rajeee requested a review from nmerket January 2, 2024 16:32
@rajeee rajeee changed the title handle empty data_point_out_json handle empty data_point_out.json Jan 2, 2024
Copy link

github-actions bot commented Jan 2, 2024

File Coverage
All files 87%
base.py 91%
exc.py 57%
hpc.py 78%
local.py 70%
postprocessing.py 84%
utils.py 91%
cloud/docker_base.py 92%
sampler/base.py 79%
sampler/downselect.py 33%
sampler/precomputed.py 93%
sampler/residential_quota.py 61%
test/shared_testing_stuff.py 85%
test/test_docker.py 33%
test/test_local.py 97%
test/test_validation.py 97%
workflow_generator/base.py 90%
workflow_generator/commercial.py 53%
workflow_generator/residential_hpxml.py 86%

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against 8b4a68f

Comment on lines 42 to 48
try:
with fs.open(filename, "r") as f:
d = json.load(f)
if not d:
return None
except (FileNotFoundError, json.JSONDecodeError):
return None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kind of curious what kind of file would run into this block. We're already handling the file being missing and it not being valid json, which is the error hit when the file is just empty. Based on messing around a little with it, I think the file would have to be just

null

I wonder where we're creating files like that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into a case where it was just "{}" when ResStock fails in some specific step.

@rajeee rajeee merged commit e860492 into develop Jan 2, 2024
6 checks passed
@rajeee rajeee deleted the empty_json branch February 3, 2024 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants