Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to database access PR #1153

Draft
wants to merge 42 commits into
base: damien/test-rds-connection
Choose a base branch
from

Conversation

robbibt
Copy link
Collaborator

@robbibt robbibt commented Dec 4, 2023

Proposed changes

Include a brief description of the changes being proposed, and why they are necessary.

Closes issues (optional)

  • Closes Issue #000
  • Closes Issue #000

Checklist (replace [ ] with [x] to check off)

  • Notebook created using the DEA-notebooks template
  • Remove any unused Python packages from Load packages
  • Remove any unused/empty code cells
  • Remove any guidance cells (e.g. General advice)
  • Ensure that all code cells follow the PEP8 standard for code. The jupyterlab_code_formatter tool can be used to format code cells to a consistent style: select each code cell, then click Edit and then one of the Apply X Formatter options (YAPF or Black are recommended).
  • Include relevant tags in the final notebook cell (refer to the DEA Tags Index, and re-use tags if possible)
  • Clear all outputs, run notebook from start to finish, and save the notebook in the state where all cells have been sequentially evaluated
  • Test notebook on both the NCI and DEA Sandbox (flag if not working as part of PR and ask for help to solve if needed)
  • If applicable, update the Notebook currently compatible with the NCI|DEA Sandbox environment only line below the notebook title to reflect the environments the notebook is compatible with

robbibt and others added 30 commits August 30, 2023 14:42
#1112)

* Major refactor of tide modelling funcs to add improved parallelization

* Fix comment

* Add missing words for spellcheck

* Add deprec warning, improve docs
* Replace deprecated xarray.open_rasterio function

* Fix replacement in test_spatial.py

* Re-run after running previous notebooks
* Update overpass notebook

* Fix S2 collection in crophealth app

* Fix interactive apps bug

* Add missing packages
…1122)

* Add glint angle func

* Add draft sunglint notebook

* Update sunglint notebook

* Black formatting

* Add word to spelllist

* Add test for sunglint func
Co-authored-by: Emma Ai <emma.ai@ga.gov.au>
Co-authored-by: Emma Ai <emma.ai@ga.gov.au>
Co-authored-by: Emma Ai <emma.ai@ga.gov.au>
…1131)

* New DEA Mangroves notebook for inclusion with DEA products notebooks

* cleared all cells and re-ran to prepare notebook for publishing

* added DEA_Mangroves to the test cube for all dates in tile x49/y24 to match the default analysis area in the mangroves notebook

* Minor edits to describe `not_observed` class in relation to the time-series analysis

* Updated index link

* Incorporates PR reviews

Thanks for the reviews @robbibt

* updated README and deleted mp4 file
* Delete DEA_products/DEA_Landsat_Surface_Reflectance.ipynb

* Add files via upload

* Add files via upload
…ith unique timesteps (#1142)

* Add analysis mode to model_tides, update tests

* Clarify which models need special definition file

* In progress update of notebook

* Update notebook, fix plot
- Provide helper functions `folium_map` and `ipyleaflet_map` that enables easy placing of an xarray Dataset on a webmap
- Provide `folium_dual_map` that lets us compare datasets side-by-side
- These functions support styling using the datacube OWS configuration

---------

Co-authored-by: uchchwhash <imam.alam@ga.gov.au>
add Aman code for colourbar for water quality notebook
water quality suspended matter notebook
remove unneeded code
minor tweak - remove unneeded code  - water quality notebook
* Pull pixel_tide resampling into function

* Identify x and y dims inside func instead

* Remove x, y dim params from function call
* Update default reviewers to remove inactive members

* Update CODEOWNERS

---------

Co-authored-by: Robbi Bishop-Taylor <Robbi.BishopTaylor@ga.gov.au>
add swinburne short course usage of dea-notebooks and tools
@robbibt
Copy link
Collaborator Author

robbibt commented Dec 4, 2023

Need to fix: Downloading_data_with_STAC.ipynb:
image

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

robbibt and others added 8 commits December 4, 2023 17:22
* Audit and update URL hyperlinks in Beginners Guide, DEA Products, How to Guides, and Interactive Maps notesbooks

* Audit and update URL hyperlinks Real World Examples notesbooks

* Rerun STAC notebook

---------

Co-authored-by: robbibt <Robbi.BishopTaylor@ga.gov.au>
@robbibt
Copy link
Collaborator Author

robbibt commented Dec 7, 2023

============================ slowest 100 durations =============================
2000.73s call     Real_world_examples/Scalable_machine_learning/1_Extract_training_data.ipynb::Cell 8
987.54s call     Real_world_examples/Change_detection.ipynb::Cell 4
844.32s call     Real_world_examples/Wetness_stream_gauge_correlations.ipynb::Cell 10
341.45s call     Real_world_examples/Urban_change_detection.ipynb::Cell 8
340.15s call     Real_world_examples/Chlorophyll_monitoring.ipynb::Cell 4
303.83s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 3
219.37s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 4
211.87s call     Real_world_examples/Coastal_erosion.ipynb::Cell 11
174.79s call     Real_world_examples/Vegetation_phenology.ipynb::Cell 5
142.60s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 7
113.96s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 12
86.34s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 8
83.31s call     Real_world_examples/Radar_water_detection.ipynb::Cell 4
66.96s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 13
64.90s call     Real_world_examples/Scalable_machine_learning/3_Evaluate_optimize_fit_classifier.ipynb::Cell 6
62.86s call     Real_world_examples/Burnt_area_mapping.ipynb::Cell 6
62.57s call     Real_world_examples/Shipping_lane_identification.ipynb::Cell 7
60.84s call     Real_world_examples/Burnt_area_mapping.ipynb::Cell 9
55.73s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 5
55.28s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 6
54.61s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 10
21.37s call     Real_world_examples/Scalable_machine_learning/5_Object-based_filtering.ipynb::Cell 2
19.35s call     Real_world_examples/Coastal_erosion.ipynb::Cell 4
17.26s call     Real_world_examples/Radar_water_detection.ipynb::Cell 6
16.77s call     Real_world_examples/Radar_water_detection.ipynb::Cell 8
16.53s call     Real_world_examples/Radar_water_detection.ipynb::Cell 12
16.29s call     Real_world_examples/Radar_water_detection.ipynb::Cell 14
15.07s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 7
14.42s call     Real_world_examples/Scalable_machine_learning/3_Evaluate_optimize_fit_classifier.ipynb::Cell 9
13.72s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 15
11.65s call     Real_world_examples/Scalable_machine_learning/5_Object-based_filtering.ipynb::Cell 8
11.16s call     Real_world_examples/Urban_change_detection.ipynb::Cell 5
10.68s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 21
10.14s call     Real_world_examples/Change_detection.ipynb::Cell 10
8.77s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 21
8.65s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 4
8.04s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 9
6.98s call     Real_world_examples/Shipping_lane_identification.ipynb::Cell 6
6.50s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 20
5.84s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 7
5.00s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 17
4.99s call     Real_world_examples/Shipping_lane_identification.ipynb::Cell 5
4.27s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 14
4.24s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 11
4.15s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 14
4.08s call     Real_world_examples/Wetness_stream_gauge_correlations.ipynb::Cell 3
4.05s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 20
4.00s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 14
3.93s call     Real_world_examples/Vegetation_phenology.ipynb::Cell 0
3.78s call     Real_world_examples/Wetness_stream_gauge_correlations.ipynb::Cell 0
3.63s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 8
3.56s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 0
3.52s call     Real_world_examples/Coastal_erosion.ipynb::Cell 6
3.44s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 0
3.43s call     Real_world_examples/Burnt_area_mapping.ipynb::Cell 0
3.40s call     Real_world_examples/Coastal_erosion.ipynb::Cell 0
2.96s call     Real_world_examples/Coastal_erosion.ipynb::Cell 8
2.91s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 19
2.84s call     Real_world_examples/Radar_water_detection.ipynb::Cell 19
2.77s call     Real_world_examples/Shipping_lane_identification.ipynb::Cell 1
2.75s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 8
2.75s call     Real_world_examples/Radar_water_detection.ipynb::Cell 16
2.73s call     Real_world_examples/Chlorophyll_monitoring.ipynb::Cell 1
2.73s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 1
2.71s call     Real_world_examples/Vegetation_phenology.ipynb::Cell 1
2.68s call     Real_world_examples/Radar_water_detection.ipynb::Cell 17
2.68s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 1
2.67s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 6
2.67s call     Real_world_examples/Radar_water_detection.ipynb::Cell 1
2.66s call     Real_world_examples/Change_detection.ipynb::Cell 1
2.64s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 8
2.58s call     Real_world_examples/Urban_change_detection.ipynb::Cell 2
2.57s call     Real_world_examples/Coastal_erosion.ipynb::Cell 1
2.57s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 1
2.57s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 1
2.56s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 29
2.54s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 3
2.52s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 1
2.52s call     Real_world_examples/Wetness_stream_gauge_correlations.ipynb::Cell 1
2.52s call     Real_world_examples/Burnt_area_mapping.ipynb::Cell 1
2.32s call     Real_world_examples/Wetness_stream_gauge_correlations.ipynb::Cell 6
2.22s call     Real_world_examples/Burnt_area_mapping.ipynb::Cell 14
2.19s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 6
2.14s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 28
2.10s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 17
2.09s call     Real_world_examples/Scalable_machine_learning/4_Classify_satellite_data.ipynb::Cell 0
2.05s call     Real_world_examples/Vegetation_phenology.ipynb::Cell 16
2.03s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 11
2.02s call     Real_world_examples/Intertidal_elevation.ipynb::Cell 14
2.00s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 9
1.96s call     Real_world_examples/Burnt_area_mapping_near_realtime.ipynb::Cell 16
1.96s call     Real_world_examples/Scalable_machine_learning/1_Extract_training_data.ipynb::Cell 4
1.95s call     Real_world_examples/Seasonal_water_extents.ipynb::Cell 0
1.94s call     Real_world_examples/Scalable_machine_learning/1_Extract_training_data.ipynb::Cell 0
1.90s call     Real_world_examples/Turbidity_animated_timeseries.ipynb::Cell 27
1.89s call     Real_world_examples/Change_detection.ipynb::Cell 0
1.88s call     Real_world_examples/Coastal_erosion.ipynb::Cell 13
1.85s call     Real_world_examples/Water_quality_suspended_matter.ipynb::Cell 0
1.84s call     Real_world_examples/Chlorophyll_monitoring.ipynb::Cell 0
1.82s call     Real_world_examples/Urban_change_detection.ipynb::Cell 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet