Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[13pt] NWS benchmark scores have dropped significantly #321

Closed
RobHanna-NOAA opened this issue Apr 9, 2024 · 0 comments · Fixed by #320
Closed

[13pt] NWS benchmark scores have dropped significantly #321

RobHanna-NOAA opened this issue Apr 9, 2024 · 0 comments · Fixed by #320
Assignees
Labels
bug Something isn't working high priority ras2fim_V2

Comments

@RobHanna-NOAA
Copy link
Collaborator

RobHanna-NOAA commented Apr 9, 2024

During recent testing of the latest code based, v2.0.2.0, and using the new run_unit_benchmark_tests.py and using eval_bench_data.py, it was discovered that nws scores dropped significantly, 10 to 20 points. The new eval_bench_data.py and some other charges already exist in WIP PR 320.

NOTE - Please build and fix against the current PR 320 / dev-inundation-fix

After considerable research, it is believed the culprit has been found. In create_geocurves.py, it applies a mask to a depth grid using the shape made from nwn_features.geometry, not considering/not seeing donut holes in the middle of some inundation depth grids. This is a known issue but has proven to have far more impact than expected on nws/ahps sites due to the small spatial area.

There may also be a problem with the geometry based on the max depth grid, but more research is required.

The depth grids appear to be correct but they are not being used directly against the rating curve records. It appears that if we start with the starting rating curve, with each record in it, find the matching depth grid from the 05_hecras_outputs folder, then use the geometry of that to add to the newly merged geocruves.csv. The profile_number in each record in the rating curve, always matches the correct depth using the following naming convention as an example based on 12090301.

  • starting with r06_create_rating_curves\12365_COLORADO RIVER\rating_curve_5791848.csv -> profile_num of 93 (for example)
  • find 05_hecras_output\12368_COLORADO RIVER\COLORADO RIVER\Depth (flow93_ft).12368.12368.tif (notice profile number of 93 and the 93 in the depth grid name). This is the geometry we need for that record in the final geocurve.

PS.. the vertical bars in this bar chart are a temp bug to be fixed later in eval_bench_data.py:

Image

Below, the pink represents donut holes that are being retained by the geocurve and result in over inundation.

Image

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working high priority ras2fim_V2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant