Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add uat option to validator script and document in readme #779

Merged
merged 4 commits into from May 2, 2024

Conversation

niarenaw
Copy link

Purpose

  • Update dswx-s1-validator code to address R3 calval use cases

Proposed Changes

  • [ADD] endpoint option to specify OPS or UAT CMR venue
  • [UPDATE] README to address new argument
  • [UPDATE] README to fix iso timestamps in example

Testing

  • Code has been used with R3 calval data

@niarenaw niarenaw requested a review from riverma March 29, 2024 19:24
@hhlee445 hhlee445 added the pcm.r03 PCM Release 3 label Mar 29, 2024
@riverma
Copy link
Collaborator

riverma commented Mar 30, 2024

Thanks a ton @niarenaw for your contribution! Let me test this out against some of our existing test cases and then get to approving / merging 👍

Copy link
Collaborator

@riverma riverma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution @niarenaw - excellent!

Couple suggestions inline but two I wanted to add:

  • Did you make the regex fix to allow the S1B RTC granules to work? If so could you contribute that to this PR and rename the PR / purpose accordingly?
  • UAT CMR has less granules than OPS, and I noticed that many times no granules are returned. I think we should add a sys.exit(1) right after this line since now this is a common occurrence. Would you consider adding that?


### Usage Examples

* Retrieve a list of MGRS Tile Set IDs for the RTC burst processing a given time range on CMR, and filter the results to show only MGRS Tile Sets that had coverage of greater than or equal to 50%.

```
$ python dswx_s1_validator.py --start "2023-12-05T01:00:00Z" --end "2023-12-05T03:59:59Z" --db MGRS_tile_collection_v0.2.sqlite --threshold 50
Querying CMR for time range 2023-12-05T01:00:00Z to 2023-12-05T03:59:59Z.
Querying CMR for time range 2023-12-05T01:00:00 to 2023-12-05T03:59:59.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep the Z as that denotes times are expected to be in UTC.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get a ValueError, when including Z: ValueError: Invalid isoformat string: '2023-12-05T01:00:00Z'

report/dswx-s1-validator/dswx_s1_validator.py Show resolved Hide resolved
report/dswx-s1-validator/dswx_s1_validator.py Show resolved Hide resolved
@niarenaw
Copy link
Author

niarenaw commented Apr 4, 2024

@riverma thanks for you suggested changes -- I've added them.

@hhlee445
Copy link
Contributor

@niarenaw @riverma i'm trying to merge this PR, however, there are some conflict. @riverma can you help to resolve the conflicts?

…d failures

- Added default value to timestamp arg
- Better logging for failure analysis
@riverma
Copy link
Collaborator

riverma commented May 2, 2024

Thanks @niarenaw for the additions! I've made a couple improvements to enhance the reliability. Was getting a number of granule query batch failures, but I think that's fixed now. Will merge. Thanks @hhlee445 as well.

Copy link
Collaborator

@riverma riverma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @niarenaw for your help improving this script!

@riverma riverma merged commit a2f0999 into issue-703 May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pcm.r03 PCM Release 3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants