Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thredds palettes #7059

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
e09266a
read list of palettes from Thredds server layer metadata service
sixlighthouses Feb 8, 2024
33e4694
temp fix for using availablePalettes trait
sixlighthouses Feb 9, 2024
ad99869
use 2 new traits for Thredds Palettes in WebMapServiceCatalogItemTrai…
sixlighthouses Feb 23, 2024
f00d68a
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Feb 23, 2024
7e2b20e
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Mar 12, 2024
1db060e
merge main and update CHANGES
sixlighthouses Mar 12, 2024
9eefa33
Update lib/Models/Catalog/Ows/WebMapServiceCapabilitiesStratum.ts
sixlighthouses Mar 14, 2024
1a477f4
Update lib/Models/Catalog/Ows/WebMapServiceCapabilitiesStratum.ts
sixlighthouses Mar 14, 2024
1227d07
Update lib/Models/Catalog/Ows/WebMapServiceCapabilitiesStratum.ts
sixlighthouses Mar 14, 2024
ecabd69
fix prettier issue
sixlighthouses Mar 15, 2024
dc1a2b5
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Mar 15, 2024
3e23262
remove superfluous anon function
sixlighthouses Mar 19, 2024
e9209ff
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Mar 19, 2024
54b4420
if no palette URL is in the style abstract resolve and return [] rath…
sixlighthouses Mar 20, 2024
1d22d63
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Mar 28, 2024
843ab42
remove thredds prefix from traits name, use the uri.clone method to g…
sixlighthouses Apr 2, 2024
086f24b
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Apr 7, 2024
edc06b2
wip
sixlighthouses Apr 9, 2024
38c8c9d
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Apr 9, 2024
a14d46e
move initial fetchPalettes into forceLoadMetadata
sixlighthouses Apr 10, 2024
0fd38b8
palettes correctly persisting after style change
sixlighthouses Apr 12, 2024
9f91a7d
wip
sixlighthouses Apr 12, 2024
41892d2
correct the check for undefined default style and remove console.logs
sixlighthouses Apr 14, 2024
722db12
run prettier
sixlighthouses Apr 14, 2024
2528d60
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses Apr 17, 2024
bacefe0
add CHANGES entry
sixlighthouses Apr 18, 2024
0dfd466
Merge branch 'main' into thredds-palettes
sixlighthouses May 7, 2024
283bda1
Merge branch 'main' into thredds-palettes
sixlighthouses May 7, 2024
f8d87f0
wip
sixlighthouses May 9, 2024
c07fb88
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses May 9, 2024
e8fc7b6
wip
sixlighthouses May 10, 2024
2ee6c8f
wip
sixlighthouses May 12, 2024
d8d11b6
correct the traits for new loadable stratum NCWMSGetMetaDataStratum
sixlighthouses May 13, 2024
2ddd71c
remove unused imports
sixlighthouses May 13, 2024
d06645c
Merge remote-tracking branch 'origin/main' into thredds-palettes
sixlighthouses May 13, 2024
545cf4b
wip
sixlighthouses May 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

#### next release (8.6.1)

- Add ability for users to select Thredds palettes when available from Thredds servers for changing the visualisation of data.

- [The next improvement]

#### 8.6.0 - 2024-03-12
Expand Down
6 changes: 6 additions & 0 deletions lib/Models/Catalog/Ows/WebMapServiceCapabilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ export type CapabilitiesDimension = string & {
readonly current?: boolean;
};

export type Palettes = {
palettes: ReadonlyArray<string>;
defaultPalette: string;
scaleRange: number[];
};

export type CapabilitiesExtent = string & {
readonly name: string;
readonly default?: string;
Expand Down
91 changes: 85 additions & 6 deletions lib/Models/Catalog/Ows/WebMapServiceCapabilitiesStratum.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ import WebMapServiceCapabilities, {
getRectangleFromLayer
} from "./WebMapServiceCapabilities";
import WebMapServiceCatalogItem from "./WebMapServiceCatalogItem";
import { Complete } from "../../../Core/TypeModifiers";

const dateFormat = require("dateformat");

Expand Down Expand Up @@ -89,6 +90,62 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
) as this;
}

async fetchPalettes(
abstract: string,
legend:
| Complete<{
title?: string | undefined;
url?: string | undefined;
paletteUrl?: string | undefined;
imageScaling?: number | undefined;
urlMimeType?: string | undefined;
items?:
| Complete<{
title?: string | undefined;
multipleTitles?: string[] | undefined;
maxMultipleTitlesShowed: number | undefined;
titleAbove?: string | undefined;
titleBelow?: string | undefined;
color?: string | undefined;
outlineColor?: string | undefined;
outlineWidth?: number | undefined;
multipleColors?: string[] | undefined;
imageUrl?: string | undefined;
marker?: string | undefined;
rotation: number | undefined;
addSpacingAbove?: boolean | undefined;
imageHeight: number | undefined;
imageWidth: number | undefined;
}>[]
| undefined;
backgroundColor?: string | undefined;
}>
| undefined
): Promise<StratumFromTraits<WebMapServiceAvailableStyleTraits>[]> {
const paletteResult: StratumFromTraits<WebMapServiceAvailableStyleTraits>[] =
[];

const urlRegex = /(https?:\/\/[^\s]+)/g;
const urls = abstract?.match(urlRegex);

if (urls) {
const paletteUrl = proxyCatalogItemUrl(this.catalogItem, urls[0]);
const response = await fetch(paletteUrl);
const data = await response.json();
const palettes = data.palettes;
palettes.forEach((palette: any) => {
paletteResult.push({
name: palette,
title: palette,
abstract: abstract,
legend: legend
});
});
}

return paletteResult;
}

@computed get metadataUrls() {
const metadataUrls: MetadataURL[] = [];

Expand Down Expand Up @@ -172,14 +229,14 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
)?.styles;

let layerStyle: Model<WebMapServiceAvailableStyleTraits> | undefined;
let paletteUrl: string | undefined;

if (isDefined(style)) {
// Attempt to find layer style based on AvailableStyleTraits
layerStyle = layerAvailableStyles?.find(
(candidate) => candidate.name === style
);
}

// If no style is selected and this WMS doesn't support GetLegendGraphics - we must use the first style if none is explicitly specified.
// (If WMS supports GetLegendGraphics we can use it and omit style parameter to get the "default" style's legend)
if (!isDefined(layerStyle) && !this.catalogItem.supportsGetLegendGraphic)
Expand Down Expand Up @@ -250,6 +307,8 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
legendUri.setQuery("transparent", "true");
}

i;

sixlighthouses marked this conversation as resolved.
Show resolved Hide resolved
// Add colour scale range params if supported
if (
this.catalogItem.supportsColorScaleRange &&
Expand All @@ -264,7 +323,8 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
createStratumInstance(LegendTraits, {
url: legendUri.toString(),
urlMimeType: legendUrlMimeType,
imageScaling: legendScaling
imageScaling: legendScaling,
paletteUrl: paletteUrl
})
);
}
Expand Down Expand Up @@ -361,10 +421,6 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
const result: StratumFromTraits<WebMapServiceAvailableLayerStylesTraits>[] =
[];

if (!this.capabilities) {
return result;
}

const capabilitiesLayers = this.capabilitiesLayers;
for (const layerTuple of capabilitiesLayers) {
const layerName = layerTuple[0];
Expand Down Expand Up @@ -410,6 +466,29 @@ export default class WebMapServiceCapabilitiesStratum extends LoadableStratum(
});
}

if (this.catalogItem.isThredds) {
// const paletteResult: StratumFromTraits<WebMapServiceAvailableLayerStylesTraits>[] =
// [];
sixlighthouses marked this conversation as resolved.
Show resolved Hide resolved

async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this anonymous function get called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That one is an IIFE -- I used it so I can call the async fetchPalettes without having to make the parent function async

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking that one up Lawrence, it was never getting called, removed that code block

const res = result[0];
const styles = res.styles ?? null;
if (styles && styles[0].abstract) {
const palettes = await this.fetchPalettes(
styles[0].abstract,
styles[0].legend
);
console.log(palettes);
sixlighthouses marked this conversation as resolved.
Show resolved Hide resolved
result[0].styles = palettes;
return result;
} else {
return result;
}
};
} else {
return result;
}

return result;
}

Expand Down
73 changes: 71 additions & 2 deletions lib/Models/Catalog/Ows/WebMapServiceCatalogItem.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ import {
} from "../../../Table/tableFeatureInfoContext";
import WebMapServiceCatalogItemTraits, {
SUPPORTED_CRS_3857,
SUPPORTED_CRS_4326
SUPPORTED_CRS_4326,
WebMapServiceAvailableLayerStylesTraits
} from "../../../Traits/TraitsClasses/WebMapServiceCatalogItemTraits";
import CommonStrata from "../../Definition/CommonStrata";
import CreateModel from "../../Definition/CreateModel";
Expand Down Expand Up @@ -619,6 +620,72 @@ class WebMapServiceCatalogItem
}
);

fetchPalettes(): Promise<string[]> {
return new Promise((resolve, reject) => {
if (!this.isThredds) {
reject("Not a THREDDS server");
}
const urlRegex = /(https?:\/\/[^\s]+)/g;
const selectedStyle = this.availableStyles[0].styles.filter(
(style) => style.name === this.styles
);
const abstract = selectedStyle[0]?.abstract;
const urls = abstract?.match(urlRegex);
const extractedUrl = urls?.[0];

if (!extractedUrl) {
reject("No URL found in abstract");
Copy link
Contributor

@ljowen ljowen Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless we also return here the fetch below will still execute and make a request to <ur>/undefined

Should No url in the abstract be treated as an error? My understanding is it's an option property for thredds wms's so it might instead be more appropriate to resolve([])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to resolve, thanks Lawrence

}
const proxiedUrl = proxyCatalogItemUrl(this, extractedUrl!);

fetch(proxiedUrl, { method: "GET" })
.then((response) => response.json())
.then((data) => {
this.setTrait(CommonStrata.user, "threddsPalettes", data.palettes);
resolve(data.palettes);
})
.catch((error) => {
reject(error);
});
});
}

@computed
get threddsPaletteDimensions(): SelectableDimensionEnum[] {
if (!this.isThredds) {
return [];
}

const options: { name: string; id: string }[] = [];

this.threddsPalettes.map((palette) => {
options.push({
name: palette,
id: palette
});
});

return [
{
name: "Palettes",
id: `${this.uniqueId}-palettes`,
options,
selectedId: this.threddsPalette,
setDimensionValue: (
stratumId: string,
newPalette: string | undefined
) => {
runInAction(() => {
this.setTrait(stratumId, "threddsPalette", newPalette);
const currentStyle = this.stylesArray[0];
const newStyles = currentStyle.split("/")[0] + "/" + newPalette;
this.setTrait(stratumId, "styles", newStyles);
});
}
}
];
}

@computed
get styleSelectableDimensions(): SelectableDimensionEnum[] {
return this.availableStyles.map((layer, layerIndex) => {
Expand Down Expand Up @@ -671,6 +738,7 @@ class WebMapServiceCatalogItem
);
styles[layerIndex] = newStyle;
this.setTrait(stratumId, "styles", styles.join(","));
this.fetchPalettes();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a condition here to fetchPalettes is only called if the server supports it?

Currently, if you select a style for a non-THREDDs WMS - this causes an uncaught error

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry this comment doesn't make sense with all the other changes I have suggested

});
},
// There is no way of finding out default style if no style has been selected :(
Expand Down Expand Up @@ -758,7 +826,8 @@ class WebMapServiceCatalogItem
return filterOutUndefined([
...super.selectableDimensions,
...this.wmsDimensionSelectableDimensions,
...this.styleSelectableDimensions
...this.styleSelectableDimensions,
...this.threddsPaletteDimensions
]);
}

Expand Down
1 change: 1 addition & 0 deletions lib/ReactViews/Analytics/ParameterEditor.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ const ParameterEditor = createReactClass({
renderEditor() {
for (let i = 0; i < ParameterEditor.parameterTypeConverters.length; ++i) {
const converter = ParameterEditor.parameterTypeConverters[i];

const editor = converter.parameterTypeToDiv(
this.props.parameter.type,
this
Expand Down
2 changes: 1 addition & 1 deletion lib/ReactViews/Workbench/Controls/Legend.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import LegendOwnerTraits from "../../../Traits/TraitsClasses/LegendOwnerTraits";
import LegendTraits, {
LegendItemTraits
} from "../../../Traits/TraitsClasses/LegendTraits";
import Styles from "./legend.scss";
import Styles, { legend } from "./legend.scss";

/* A lookup map for displayable mime types */
const DISPLAYABLE_MIME_TYPES = [
Expand Down
8 changes: 8 additions & 0 deletions lib/Traits/TraitsClasses/LegendTraits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,14 @@ export default class LegendTraits extends ModelTraits {
})
url?: string;

@primitiveTrait({
type: "string",
name: "Palettes URL",
description:
"The URL of the endpoint which returns the list of Thredds palette"
})
paletteUrl?: string;

@primitiveTrait({
type: "number",
name: "Scaling",
Expand Down
26 changes: 26 additions & 0 deletions lib/Traits/TraitsClasses/WebMapServiceCatalogItemTraits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,16 @@ export class WebMapServiceAvailableLayerStylesTraits extends ModelTraits {
styles?: WebMapServiceAvailableStyleTraits[];
}

export class WebMapServiceAvailablePaletteTraits extends ModelTraits {
@primitiveTrait({
type: "string",
name: "URL",
description:
"The URL of the GetMetadata request for the palette. This is used to get the palettes available for a layer."
})
url?: string;
}

export class WebMapServiceAvailableDimensionTraits extends ModelTraits {
@primitiveTrait({
type: "string",
Expand Down Expand Up @@ -316,6 +326,22 @@ export default class WebMapServiceCatalogItemTraits extends mixTraits(
})
colorScaleMaximum: number = 50;

@primitiveTrait({
type: "string",
name: "Thredds palette",
description:
"palette is a non-standard property supported by THREDDS servers. This property is ignored unless WebMapServiceCatalogItem's isThredds is true. The default value is 'default'."
})
threddsPalette?: string = "default";

@primitiveArrayTrait({
type: "string",
name: "Thredds palettes",
description:
"Used to store the palettes available for a layer. This is a non-standard property supported by THREDDS servers. This property is ignored unless WebMapServiceCatalogItem's isThredds is true. The default value is ['default']."
})
threddsPalettes: string[] = ["default"];

@primitiveTrait({
type: "boolean",
name: "Use WMS version 1.3.0",
Expand Down