Add wms-group to ckan and ignore duplicate wms layers in groups

[nf-s]

What this PR does

Fixes #5745 #5893

Reverts parts of #6240

• Breaking changes:
  • CkanItemReference no longer copies default stratum to target - please use itemProperties instead.
• Revert Use CKAN Dataset name property for WMS layers as last resort.
• Add support for WebMapServiceCatalogGroup to CkanItemReference - this will be used instead of WebMapServiceCatalogItem if WMS layers can't be identified from CKAN resource metadata.
  • Add allowEntireWmsServers to CkanCatalogGroupTraits - defaults to true
• Ignore WMS Layers with duplicate Name properties

Test me

Previously broken layers

NationalMap BOM layers

• http://ci.terria.io/main/#configUrl=https%3A%2F%2Fnationalmap.gov.au%2Fconfig.json&share=s-6lVY0nbynTMbxNrcDNxAhpnoEMD
• http://ci.terria.io/wms-group-ckan/#configUrl=https%3A%2F%2Fnationalmap.gov.au%2Fconfig.json&share=s-6lVY0nbynTMbxNrcDNxAhpnoEMD

Pacificmap "GEBCO"

• http://ci.terria.io/main/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-e6pCuveWY2VQSontFWdHIB3bwFd
• http://ci.terria.io/wms-group-ckan/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-e6pCuveWY2VQSontFWdHIB3bwFd

Pacificmap "Global Distribution of Coral Reefs"

• http://ci.terria.io/main/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-yXBz4vDt6z7oFfDPBoEzsSMuJrk
• http://ci.terria.io/wms-group-ckan/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-yXBz4vDt6z7oFfDPBoEzsSMuJrk

Pacificmap "SEDAC data products and services"

• http://ci.terria.io/main/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-4DIE4HSWa5G7mqWbP6beENTTqJh
• http://ci.terria.io/wms-group-ckan/#configUrl=https%3A%2F%2Fterria-catalogs-public.storage.googleapis.com%2Fpacificmap%2Fmap-config%2Fdev.json&share=s-4DIE4HSWa5G7mqWbP6beENTTqJh

Layers which are now worse off - they are now groups

This is due to "Revert Use CKAN Dataset name property for WMS layers as last resort"

• http://ci.terria.io/main/#configUrl=https%3A%2F%2Fnationalmap.gov.au%2Fconfig.json&share=s-nxnnxNRNJKbbAPYNX2jzTID5k7M
• http://ci.terria.io/wms-group-ckan/#configUrl=https%3A%2F%2Fnationalmap.gov.au%2Fconfig.json&share=s-nxnnxNRNJKbbAPYNX2jzTID5k7M

Checklist

• There are unit tests to verify my changes are correct or unit tests aren't applicable (if so, write quick reason why unit tests don't exist)
• I've updated relevant documentation in doc/.
• I've updated CHANGES.md with what I changed.
• I've provided instructions in the PR description on how to test this PR.

[t83714]

Well done @nf-s & thanks for your excellent work -- I believe this PR gives more choices to the user and also helps to reduce the chance that we have to report an error to the user.

Also thank you for supplying the test datasets --- it makes it much easier to play with the PR 💯

re: the test datasets, I guess the first test link is the "before the PR" link and the 2nd link is the "after PR" link for the same dataset.

• I have played "NationalMap BOM layers", "Pacificmap "GEBCO", Pacificmap "Global Distribution of Coral Reefs" & Pacificmap "SEDAC data products and services". I think they all work better (using the 2nd "after PR" test link) after the PR.
• The last dataset example, unfortunately, indeed comes with a less desired outcome after the PR. I'd say generally the "name" field is meant to be a human-readable title of the resource in ckan. The chance of people using that field to supply layer name is very low as a layer name usually use - as spaces (probably why we will see more positive cases). I'd suggest asking the data custodian to fix this issue by adding a layers query parameters to the wms of the dataset. If we have to solve it, we probably need to apply some validation rules before using the "name" field as the last resort for this case. e.g. if the name field contains any space, it won't be a valid layer name. Having said that, considering the probability of this case and the correct usage of name field, I'd say probably not worth implementing it.
  • by the way, I believe the one from DGA ckan is actually an "obsolete" dataset and only shows up recently (last year) due to a data recovery issue yet to solve (by DTA now ABS). They are all prior 2018 datasets before Magda on board and crawled by DGA ckan from other sources (for this case it's adon). After 2018, those external sources are handled by Magda and those "obsolete" datasets seems to have been removed from CKAN DB but only brought back incorrectly during the data loss incident last year. Those datasets's "data_state" field in ckan package data has value "inactive". We probably can filter out in this way.
• In the end --- only a minor UI suggestion ( and probably a separate issue and should be addressed by a separate PR): When opening a share link (Take NationalMap BOM layers for example, it's correctly opened as WMS group. But as the group is added at the very bottom of the catalogue list, the user will have to scroll down before he can find the layer selection list. Do you think it will be better if we can auto-scroll the group into the viewport when it's opened via a share link?

Generally, I think this PR works very good and is ready to merge (the suggestion above probably could fit into a separate PR unless it's very tiny) and I will add my approval.

[nf-s]

Thanks @t83714 for the review!

I agree with all your points.

There is already an issue for scrolling to shared group - #5204

I have created an issue re filtering out datasets with "data_state" = "inactive" - TerriaJS/nationalmap#1143