Handle database URLs a bit better

[SpacemanPaul]

Reason for this pull request

Some downstream libraries (e.g. OWS) customise their test behaviour depending on the database configuration. They want to be able to be able to see e.g. the db username being used to connect to the database. This was not possible in the new config API if the database config was passed in as a url - Only the url was visible, forcing the client to parse the url.

Proposed changes

• If the database config is passed in as a URL, pseudo-option handlers are added to the environment for the various url component options. The values for these are then read directly from the environment url.

Note that the reverse is NOT possible because of the order in which option handlers are evaluated. However the API method datacube.cfg.psql_url_from_config is already provided for this use case.

• Closes #xxxx
• Tests added / passed
• Fully documented, including docs/about/whats_new.rst for all changes

N.B. This PR also updates whats_new.rst to be ready for an rc4 release.

[pjonsson]

I just spotted this and the typing wasn't what I expected. Is the "Any" meant to be

T = TypeVar("T")

class C:
  def validate_and_normalise(self, value: T) -> T:

to remain compatible with some old(er) Python version?

[SpacemanPaul]

I just spotted this and the typing wasn't what I expected. Is the "Any" meant to be

T = TypeVar("T")

class C:
  def validate_and_normalise(self, value: T) -> T:

to remain compatible with some old(er) Python version?

The typing could probably be stricter (as it almost always can be whenever you see an Any) but in this case the input type may not be the same type as the output type (e.g. a string may be converted to an int if required, etc.) (And the 1.9 branch requires Python 3.10+)

T = TypeVar("T")

class C:
  def validate_and_normalise(self, value: T | str | None) -> T:

would probably work, but I think the existing signature is good enough for most purposes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.80%. Comparing base (e318d1f) to head (0fb079b).

Additional details and impacted files

@@               Coverage Diff               @@
##           develop-1.9    #1575      +/-   ##
===============================================
+ Coverage        85.78%   85.80%   +0.01%     
===============================================
  Files              140      140              
  Lines            15386    15402      +16     
===============================================
+ Hits             13199    13215      +16     
  Misses            2187     2187              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Ariana-B]

LGTM