Use segment-index for ad-hoc meshing to enable rendering non-connected segments

[fm3]

The segment index introduced in #7063 could be used (if present) to find the buckets that marchingCubes for ad-hoc meshing (aka isosurface routes) needs to traverse. This could then replace the edge-pixel heuristic that is currently used to determine when marchingCubes stops.

[fm3]

This is about the tracingstore route /volume/:tracingId/isosurface. It comptues the isosurface (=mesh) via the MarchingCubes algorithm for a segment id in a given subvolume, and returns it to the frontend along with a list of neighbors of the volume, where the segment “touched the walls”.

The frontend then typically continues by requesting the isosurface for the same segment id, but for those neighbor volumes.

This heuristic yields incomplete meshes when the segment is not connected.

Now using the segment index (if it exists it can be acquired by VolumeSegmentIndexService.getSegmentToBucketIndexWithEmptyFallbackWithoutBuffer), the frontend could instead get the full list of buckets where the segment lies, with a separate route, and then request all of those. (or possibly larger subvolumes, I think larger requests than 32³vx make sense).

If you get to this, maybe you could coordinate with @philippotto what a sensible protocol for this may be.

Note that the segment index does not always exist. The add-hoc meshing for on-disk segmentations should also not be broken (there is a route in the datastore as well, which ultimately uses the same IsosurfaceService)

@valentin-pinkau also mentioned that this non-connected heuristic may actually be what the user wants in some cases (e.g. all synapses have the same segment id, but the user is interested in the mesh of only one synapse). I’m not sure if this should be configurable or if the fault here lies with the user who “abuses” what a segment id is? Any opinions welcome :)

[philippotto]

If you get to this, maybe you could coordinate with @philippotto what a sensible protocol for this may be.

Sure, we can do that. Just as a short note: This API could probably look like a slimmed-down variant of the precomputed mesh case. There, a first request to get the available chunks for a segment id is also sent (compare route ${dataStoreUrl}/data/datasets/${datasetId.owningOrganization}/${datasetId.name}/layers/${layerName}/meshes/formatVersion/3/chunks).

Note that the segment index does not always exist. The add-hoc meshing for on-disk segmentations should also not be broken (there is a route in the datastore as well, which ultimately uses the same IsosurfaceService)

Ok, this means that the heuristic with segments "touching the walls" will stay for the case where we don't have the segment statistics, right?

[philippotto]

@valentin-pinkau also mentioned that this non-connected heuristic may actually be what the user wants in some cases (e.g. all synapses have the same segment id, but the user is interested in the mesh of only one synapse). I’m not sure if this should be configurable or if the fault here lies with the user who “abuses” what a segment id is? Any opinions welcome :)

I'd vote for showing all non-connected meshes, nevertheless. If the user wants them separated, different ids should be used (by the user). In general, it's better to show too much data than too little, I'd argue.
Later iterations could show the non-connected components in the segments tab as an additional hierarchy layer, but I'd wait until this really becomes necessary.

[fm3]

Ok, this means that the heuristic with segments "touching the walls" will stay for the case where we don't have the segment statistics, right?

I’d say so, yes.

If the user wants them separated, different ids should be shown

Not sure I understand what you mean. Can the frontend find out how to split them?

[philippotto]

Not sure I understand what you mean. Can the frontend find out how to split them?

Sorry, I used the wrong word. I meant "different ids should be used by the user".

[frcroth]

Backend: Create a route in tracingstrore to expose segment index
- Do not return buckets as is, but something less granular -> segmentId, tracingId, mag, cubeSize:Vec3Int parameter for the resolution of buckets where that segment id occurs
Frontend: Use route to query ad-hoc meshing multiple times.