Tracker: 4.3 release

[ibgreen]

Also see the following sub trackers

• Tracker: @loaders.gl/core #361
• Tracker: Point Cloud Usability and Polish #536
• Tracker: @loaders.gl/potree #212
• Tracker: FlatGeobuf Geospatial Loader #716
• Tracker: textures module #747
• tracker: GIFBuilder #765
• Tracker: RequestScheduler Roadmap #754
• CSVLoader upgrade #1437
• Tracker: ArrowJS docs punch list #209
• tracker: Arrow Schema support #1443

v4.3

• Consolidation
  • Replace DataViewFile and FileHandleFile with ReadableFile P0 @dariaterekhova-actionengine
• Loader upgrades
  • Laz-Perf v0.0.6 @ibgreen (check if needed to restore tests)
  • Basis v1.16.4 @belom88 https://github.com/BinomialLLC/basis_universal/releases
  • Upgrade to latest flatgeobuf update flatgeobuf dependency #2459
  • Upgrade to latest geopackage
  • Use fflate instead of pako P2 WIP: feat(compression): {Gzip,Deflate}Compression now use fflate instead of pako. #2474
• Typing and doc improvements (continued from 4.0)
  • Add Types to DracoWriter P1
  • Add types to ShapefileLoader P0 Tracker: 4.3 release #2423 feat(shapefile): Typed ShapefileLoader #2611
  • Firm up GeoJSONLoader shapes
  • Firm up CSVLoader shapes
  • Firm up ShapefileLoader shapes
• Schema improvements
  • Switch from Arrow Schemas to "Transferable" Schemas P0
• Draco
  • See if we can enable EDGE_BREAKER compression in draco @belom88 P1
  • Measure size of loaded draco geometries.
• Examples (more Cloud-Native Geospatial direct access to very large archives)
  • https://github.com/OvertureMaps/data/blob/main/README.md

Typed Loaders

• ArrowLoader
  • ColumnarTable type, align with schema module type
• ArrowWriter
  • ColumnarTable type, align with schema module type
• BSONLoader
• BSONWriter
• CSVLoader
  • More specific table type?
• CSVWriter
  • More specific table type?
• DracoLoader
• DracoWriter
• ExcelLoader
• FlatGeobufLoader
• GeopackageLoader
• GeotiffLoader
• GLBLoader
• GLBWriter
• GLTFLoader
• GLTFWriter
• PLYLoader
• [ ]

v4.0 Follow-up

• Unbreak Node Workers
  • yarn test node-debug broken
  • Node Workers broken during test on 4.0 #2735 P0 @belom88
  • Restore all tests https://github.com/search?q=repo%3Avisgl%2Floaders.gl+%22TODO+v4%22&type=code
• Tests
  • Test Website
  • Test all examples
  • Update bundler examples
  • Investigate / Restore dist tests (ocular-test dist)
  • Update CONTRIBUTING and developer guides.
  • Bump in luma.gl v9
  • Bump in deck.gl v9
  • Bump in deck.gl v8.10
• Docs and Cleanup
  • Document any changes to shapes
  • Update release notes to cover action engine work P0 @dsavinov-actionengine
  • Document polyfills updates for Node v18 and v20 @ibgreen

v5.0

• Capabilities
  • Support batched workers P1
  • Rebase WorkerLoaders on @loaders.gl/worker-utils - P1
• Features
  • Official WMS Source/Service API P1
  • Experimental Geotiff, Zarr, Parquet, ... loaders

Deprecations

• Helper functions like isPureObject should be exported from @loaders.gl/loader-utils rather than @loaders.gl/core #3001
• WMS createImageService etc should be removed.

We want to be able to return (optional) arrow schemas from geospatial loaders

• Update GIS category docs
• Add types for GIS data format (return type for GIS loaders)
• Update Shapefile loader
• Update KML, GPX, TCX
• ...

Background

The following loaders are listed in https://loaders.gl/docs/specifications/category-gis

gis object:

• schema?:
• data: features, binary etc.
• encoding?: string
• projection?: string
• loaderData: object - Loader specific data`

shapefile specific

• prj?: string
• shx: ...
• header: shapeHeader,

| Loader | geojson | binary | raw | batch | comments |
| --- | --- | --- | --- |
| GPXLoader | a FeatureCollection | attributes object | parsed XML |
| KMLLoader | a FeatureCollection | attributes object | parsed XML |
| TCXLoader | a FeatureCollection | attributes object | parsed XML |
| MVTLoader | layers with FeatureCollections |
| GeoJSONLoader |
| ShapefileLoader | gis object | gis object | - | gis object |
| SHPLoader |
| FlatGeobufLoader | a FeatureCollection | - | - |
| GeoPackageLoader | layers with FeatureCollections | - | - |
| WKBLoader | a single geojson geometry (not feature) | - | - |
| WKTLoader | a single geojson geometry (not feature) | - | - |

Tables category

| Loader | json | binary | raw | comments |
| --- | --- | --- | --- |
| DBFLoader |

Problem: Atomic loaders

Currently

• Where to put schema?
• Change return values?
• Add metadata option that returns a wrapper object?

Problem: Non binary loaders

Focus on binary loaders. Omit schema from non-binary loaders for now

Problem: Batched Loaders

• Schema: There is already support for an optional metadata batch, that could accommodate the schema.
• Layers: ???

Roadmap

• Binary support for WKT / WKB?