Cloud optimised geotiff main

[staffordsmith83]

What this PR does

Fixes https://github.com/TerriaJS/product/issues/156

This is a protected 'main' branch for all COG support tickets.
This should eventually be merged into terriajs/main

Test me

This should open the branch with a catalog of COGs to test:

http://ci.terria.io/cloud-optimised-geotiff-main/#clean&https://gist.githubusercontent.com/staffordsmith83/b277bb86fc5902d924e5d3cf8ae91146/raw/b8461bbe6b1cf2d1054d04faa3baa21ac7d66c67/cog-examples-2.json

Try the example COGs provided in 2D and 3D mode.

@update 1st September 2023

Current Blockers and Way Forward

Problem 1

We are still using two different mechanisms for displaying COG data in 2D and 3D mode. In 2D we use georaster-layer-for-leaflet as a dependency. In 3d mode we have adapted the code from TIFFImageryProvider and included it in our lib/ThirdParty directory in terriajs.

This is not ideal, as new features require double the development time. This is due to the fact that leaflet only works in Web Mercator Tiling Scheme, while Cesium is more flexible. I have had numerous discussions with Kevin as to how to better implement this - he thinks the best approach would be to resample non-native datasets onto canvas elements aligned with Web Mercator tiling scheme. He thinks that we could use the approach from the georaster-layer-for-leaflet code. He acknowledges this is a significant amount of work.

For now we have used an approach novel in the terria codebase - MappableMixin instances now have an optional property method overrideCreateLeafletLayer which takes the imagery provider and a clipping rectangle and returns a TerriaLeafletLayer.

NB: TerriaLeafletLayer is a union type of the two previously used Leaflet layer types, and a new one for COGs, GeorasterTerriaLayer:

export type TerriaLeafletLayer =
  | GeorasterTerriaLayer
  | ImageryProviderLeafletGridLayer
  | ImageryProviderLeafletTileLayer;

To backtrack, overrideCreatLeafletLayer can be defined for the catalog items that need a different method for creating the leaflet layer vs the cesium imagery provider, such as in CogCatalogItem.ts and CogCompositeCatalogItem.ts. The presence of this function is detected in Leaflet._createImageryLayer here:

terriajs/lib/Models/Leaflet.ts

Line 132 in 31f2ecc

if (overrideCreateLeafletLayerFn) {

Whilst this implementation works, there are a few issues:

• need to define functionality like pickFeatures twice - e.g. in TIFFImageryProvider and also in GeorasterTerriaLayer
• If a user loads a COG in 2D mode and switches to 3D mode (or vice-versa), the dataset will be requested again from the server/bucket
• The behaviour is different in 2D and 3D mode, which may not be ideal for user experience
• Implementing cog-composite is possible in the 2D code pathway, but not in 3D code pathway yet...

Problem 2

georaster-layer-for-leaflet has a few issues, discontinuities when displaying dataset in non-native coord system(see https://github.com/TerriaJS/product/issues/304 and sharelink with example here), and single-band COGs not being coloured correctly (see https://github.com/TerriaJS/product/issues/306) .

There has been a very recent status update from Daniel Dufour, the developer, that he is working on a new version and has apologised it is taking so long. But he says that the technical approach has been validated and the rendering algorithms are already complete

Problem 3

The reprojection method employed by the 3d library TIFFImageryProvider works for many projections, but is theoretically flawed and may not work for complex projections such as those over the poles. Kevin has suggested that reprojecting COGs on the fly is not something that can be done both easily and rigorously in Cesium without changes to cesium.js.

Kevin draws our attention to a large and now abandoned PR where someone (a Cesium employee) attempted to add support for arbitrary imagery projections: CesiumGS/cesium#7438

He also says: "A tiling scheme always has to be a regular quadtree subdivision. In theory the coordinate space subdivided can be based on any projection, but the rest of Cesium only knows how to deal with longitude/latitude. This is definitely fixable. Basically, to map an image with an arbitrary projection onto a terrain tile, we need to 1) figure out every tile in the imagery provider's tiling scheme that overlaps with the terrain tile at all. 2) compute a set texture coordinates for the terrain tile based on the imagery provider's projection, and 3) use those texture coordinates to sample a rectangle texture in that imagery provider's projection. Effectively this gets the 3D hardware to do a "warp" of the texture on the fly. It effectively uses the terrain vertices as a warp grid. And it's basically free because this is just normal texture mapping that we have to do anyway, even for "normal" projections.
So that's all doable, I think, but it will take quite a bit of work to implement it in CesiumJS."

The developer of TIFFImageryProvider is trying to improve his reprojection system: See this ticket: hongfaqiu/TIFFImageryProvider#16

UPDATE: Te developer has now flagged reprojection as experimental, as he cannot find a way to do the transformations accurately without CPU overload: hongfaqiu/TIFFImageryProvider#16 (comment)

Way Forward

Whilst this branch is working currently, it is certainly desirable to have a single code pathway for 2D and 3D mode. In conversation with @nf-s we think that using georaster-layer-for-leaflet as the primary library for handling COGs might be the best, as Leaflet is fussier compared to Cesium, and that library offers some more functionality than TIFFImageryProvider, and will integrate with geoblaze for more client side analysis. The developer of TIFFImageryProvider has also lost confidence in its reprojection method.

The challenge will be to find a way to display the resulting Leaflet layer on the Cesium globe. It may be useful to generalise this, and find a way to add Leaflet layers to the Cesium globe in general, to allow us to take advantage of the extensive Leaflet plugin ecosystem.

This development should wait until Daniel Dufour releases the new version of georaster-layer-for-leaflet, as it sounds like this will be a major release and may include significant changes.

Getting there and its encouraging to see COGs in Terria!

Checklist

• There are unit tests to verify my changes are correct or unit tests aren't applicable (if so, write quick reason why unit tests don't exist)
• I've updated relevant documentation in doc/.
• I've updated CHANGES.md with what I changed.
• I've provided instructions in the PR description on how to test this PR.

[AnaBelgun]

@na9da to do a test run

[imakihi]

This is awesome. We are going to test this!
Thank you for your hard work!!

@staffordsmith83
Can you tell me how to use the cog composite?

I tested 50GB and 150GB files and the 50GB image showed nicely and no stress. 150GB one must be too big, so a terria instance crashed when I tried to show it.

Thank you!

[staffordsmith83]

This is awesome. We are going to test this! Thank you for your hard work!!

@staffordsmith83 Can you tell me how to use the cog composite?

I tested 50GB and 150GB files and the 50GB image showed nicely and no stress. 150GB one must be too big, so a terria instance crashed when I tried to show it.

Thank you!

Glad you like it!

This is an example json for a cog-composite catalog item. Please note, this only works in 2D mode currently:

    {
      "name": "COG Composite Test",
      "type": "cog-composite",
      "bands": [
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band04.tif",
          "bandName": "red"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band03.tif",
          "bandName": "green"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band02.tif",
          "bandName": "blue"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band08.tif",
          "bandName": "nir_1"
        }
      ],
      "style": [
        {
          "functionName": "trueColour",
          "functionInputs": ["red", "green", "blue"]
        },
        {
          "functionName": "ndvi",
          "functionInputs": ["red", "nir_1"]
        }
      ]
    },

Please note that this branch is still in development, and there are some errors in how we reproject non-native coordinate systems.

Could you provide a link to the 150gb COG that crashed Terria please? That would be helpful thanks!

[imakihi]

@staffordsmith83
I tested the cog-composite after I downloaded, reprojected to wgs84, and uploaded to my S3 bucket all 4 images you listed in your example but it didn't work. My terria instance kept crashing with the "Error code: Out of Memory" message.

After I clipped all images to a small extent then I could make it work. But the composit layer looked black and white. Do I need to set up something to make a true color or ndvi composite layer except my catalog setting?

Thanks!

Hiroo

[staffordsmith83]

Does it work when pointing to the image links in the example?

[imakihi]

@staffordsmith83
Yes, I can see your example item.
Your example also looks black and white to me. I see a style setting in your catalog entry but I may don't know how to intract with those two settings, true color and NDVI styles. Can I chose those settings intaractively?

Thank you!

Hiroo

[staffordsmith83]

Dear Hiroo yes this feature is really just experimental at the moment, but it’s great that you are interested! I am away until the 31st of August and will work through it with you then, thanks for your help

On Tue, 22 Aug 2023 at 3:09 am, Hiroo Imaki ***@***.***> wrote: @staffordsmith83 <https://github.com/staffordsmith83> Yes, I can see your example item. Your example also looks black and white to me. I see a style setting in your catalog entry but I may don't know how to intract with those two settings, true color and NDVI styles. Can I chose those settings intaractively? Thank you! Hiroo — Reply to this email directly, view it on GitHub <#6833 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44PP7HN54K4U3CZQHX6PDXWOXAHANCNFSM6AAAAAA3ATCMSM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- S

[staffordsmith83]

Dear @imakihi I am back in the office now and I have pushed a commit to fix cog-composite catalog items. The example in this sharelink should work now, I am just waiting for it to build, and will check again tonight!

Please specify a cog-composite catalog item like this:

{
      "name": "COG Composite Test",
      "type": "cog-composite",
      "bands": [
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band04.tif",
          "bandName": "red"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band03.tif",
          "bandName": "green"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band02.tif",
          "bandName": "blue"
        },
        {
          "url": "https://dea-public-data.s3.ap-southeast-2.amazonaws.com/baseline/ga_s2am_ard_3/55/HFB/2020/06/18/20200618T032225/ga_s2am_nbart_3-2-1_55HFB_2020-06-18_final_band08.tif",
          "bandName": "nir_1"
        }
      ],
      "style": [
        {
          "functionName": "trueColour",
          "functionInputs": ["red", "green", "blue"]
        },
        {
          "functionName": "ndvi",
          "functionInputs": ["red", "nir_1"]
        }
      ]
    },

Currently, we only have a few style functions defined in colourMappings.ts - I would suggest experimenting with trueColour and ndvi as in the example.
You need to specify an array of objects with a url and a bandName for each of the bands you need: i.e. if you just want true colour, you only need to specify red green and blue bands.
We are still working on this featue and very happy for you to test for us!

thanks,
Stafford.

[imakihi]

@staffordsmith83
Thank you for keep working on this important function!
So I have one additional question regarding a value picking in the 3D mode.
I converted one of my dem to COG and showed it in Terria. I clicked several locations on the layer in the 3D mode but the values I got were different than I expected. I created the COG file based on a DEM tiff so I expected I could see elevation values when I clicked somewhere within the layer.

I think I could get correct values in the 2D mode though...

Can you tell me what kind of values I was getting? In the following image, whiter areas should have higher elevation values...

Thank you!!!

[staffordsmith83]

Dear @imakihi could you please provide a link to the dataset? A link to s3 bucket maybe, so I can test? Also, if you are interested, please see the edited description for this ticket, it describes where we are at with development of this branch, and our path forward

[imakihi]

@staffordsmith83 and all
Just in case here is the data I uploaded.
{
"id":"cog003",
"type":"cog",
"name":"shizuoka test ground elevation",
"url":"https://s3.ap-northeast-1.amazonaws.com/3dimension.jp/tmp/imaki_test/shizuoka_ground_test.tif"
},
Thank you.