Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup #1406

Merged
merged 17 commits into from May 4, 2023
Merged

Cleanup #1406

merged 17 commits into from May 4, 2023

Conversation

Ariana-B
Copy link
Contributor

@Ariana-B Ariana-B commented Feb 17, 2023

Reason for this pull request

The datacube-core code could use some cleanup. I'm starting off with some small/easy wins, primarily focused on removing/consolidating duplicated logic and making code more pythonic and concise.

Proposed changes

  • Merged get_doc_offset and get_doc_offset_safe

  • Prettified DocReader properties, and added doc attribute

  • Added a few deprecation warnings with the deprecat library

  • Closes #xxxx

  • Tests added / passed

  • Fully documented, including docs/about/whats_new.rst for all changes


📚 Documentation preview 📚: https://datacube-core--1406.org.readthedocs.build/en/1406/

@SpacemanPaul
Copy link
Contributor

All looks good on a quick glance. Will review properly when it's ready - thanks for all this, Ariana.

@Ariana-B Ariana-B marked this pull request as ready for review February 20, 2023 23:54
@Ariana-B Ariana-B force-pushed the cleanup_and_deprecation_warnings branch from c59b138 to bf1faf3 Compare February 22, 2023 23:30
Copy link
Contributor

@SpacemanPaul SpacemanPaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to getting rid of those DatasetTypes!

@@ -21,9 +21,11 @@
schema_validated, DocReader
from datacube.index.eo3 import is_doc_eo3
from .fields import Field, get_dataset_fields
from ._base import Range, ranges_overlap # noqa: F401
from ._base import Range
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ranges_overlap is here so you can from datacube.model import ranges_overlap.

You can get rid of the flake8 ignore by defining an __all__

except (AttributeError, KeyError, ValueError):
continue
return fields
return {name: field.extract(self.doc)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there implications for not catching those errors? It feels like a risky thing to move, but it seems things would have to be pretty broken elsewhere for any of those errors to occur.

Copy link
Contributor Author

@Ariana-B Ariana-B Mar 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There'd have to be a mismatch between the MetadataType definition and the Dataset's metadata document. If that doesn't already cause other problems elsewhere, it might be better to throw an exception or at least log a warning instead of letting it silently slip past.

@Ariana-B Ariana-B force-pushed the cleanup_and_deprecation_warnings branch from 41285d6 to d8e8057 Compare May 3, 2023 04:53
@Ariana-B Ariana-B changed the base branch from develop to develop-1.9 May 3, 2023 06:39
@codecov
Copy link

codecov bot commented May 3, 2023

Codecov Report

Patch coverage: 95.65% and project coverage change: +0.01 🎉

Comparison is base (5fcbe21) 91.95% compared to head (a12ab1a) 91.97%.

Additional details and impacted files
@@               Coverage Diff               @@
##           develop-1.9    #1406      +/-   ##
===============================================
+ Coverage        91.95%   91.97%   +0.01%     
===============================================
  Files              135      135              
  Lines            15085    15070      -15     
===============================================
- Hits             13872    13860      -12     
+ Misses            1213     1210       -3     
Impacted Files Coverage Δ
datacube/api/core.py 95.64% <ø> (ø)
datacube/api/grid_workflow.py 97.05% <ø> (ø)
datacube/index/postgres/index.py 98.94% <ø> (ø)
datacube/model/properties.py 66.41% <ø> (+0.49%) ⬆️
datacube/model/utils.py 99.39% <ø> (ø)
datacube/utils/__init__.py 100.00% <ø> (ø)
datacube/scripts/ingest.py 83.83% <75.00%> (ø)
datacube/utils/documents.py 98.61% <94.73%> (+0.25%) ⬆️
datacube/drivers/postgis/_fields.py 90.67% <100.00%> (ø)
datacube/drivers/postgres/_fields.py 90.53% <100.00%> (ø)
... and 6 more

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@SpacemanPaul SpacemanPaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All makes sense to me. Thanks again, Ariana.

@Ariana-B Ariana-B merged commit 001870f into develop-1.9 May 4, 2023
20 checks passed
@Ariana-B Ariana-B deleted the cleanup_and_deprecation_warnings branch May 4, 2023 01:11
@Ariana-B Ariana-B self-assigned this Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants