Cannot apply affine transform

[ziw-liu]

When executing mantis apply-affine with a transform estimated with mantis estimate-affine, the CLI crashes:

(shrimpy) /hpc/projects/intracellular_dashboard/viral-sensor/2024_02_04_A549_DENV_ZIKV_timelapse mantis apply-affine -s 1-reconstruct/2024_02_04_63X_timelapse_2.zarr/B/4/0 -t 0-convert/2024_02_04_63X_timelapse_2.zarr/B/4/0 -o 2-register/test.zarr -c 2-register/Phase-mCherry-B_4_0.yaml

Input positions: ['1-reconstruct/2024_02_04_63X_timelapse_2.zarr/B/4/0']
Output position: 2-register/test.zarr

REGISTRATION PARAMETERS:
Transformation matrix:
[[ 1.00000000e+00  0.00000000e+00  0.00000000e+00 -7.00000000e+00]
 [ 0.00000000e+00  9.99994278e-01  3.36904102e-03 -9.08547401e+00]
 [ 0.00000000e+00 -3.36904102e-03  9.99994338e-01 -6.10388231e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]
Voxel size: [0.25       0.41199998 0.41200001]

Finding largest overlapping volume between source and target datasets
Starting Largest interior rectangle (LIR) search
Shape of cropped output dataset: (77, 1024, 1024)

Function to be applied:         <function apply_affine_transform at 0x1542073dfe20>
Input data path:        1-reconstruct/2024_02_04_63X_timelapse_2.zarr/B/4/0
Output data path:       2-register/test.zarr
 Input data tree: /
 ??? 0 (48, 1, 77, 1024, 1024) float32


Starting multiprocess pool with 1 processes
Processing c=0, t=0
input_channel_indices: [0]
Finished Writing.. t=0
Processing c=0, t=24
input_channel_indices: [0]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "//envs/shrimpy/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/multiprocessing/pool.py", line 51, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//shrimPy/mantis/cli/utils.py", line 231, in apply_transform_to_zyx_and_save_v2
    output_dataset[0].oindex[t_idx, output_channel_indices] = transformed_czyx
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/zarr/indexing.py", line 695, in __setitem__
    return self.array.set_orthogonal_selection(selection, value, fields=fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/zarr/core.py", line 1571, in set_orthogonal_selection
    indexer = OrthogonalIndexer(selection, self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/zarr/indexing.py", line 626, in __init__
    dim_indexer = IntDimIndexer(dim_sel, dim_len, dim_chunk_len)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/zarr/indexing.py", line 163, in __init__
    dim_sel = normalize_integer_selection(dim_sel, dim_len)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/zarr/indexing.py", line 135, in normalize_integer_selection
    raise BoundsCheckError(dim_len)
zarr.errors.BoundsCheckError: index out of bounds for dimension with length 2
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "//envs/shrimpy/bin/mantis", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//shrimPy/mantis/cli/apply_affine.py", line 159, in apply_affine
    process_single_position_v2(
  File "//shrimPy/mantis/cli/utils.py", line 393, in process_single_position_v2
    p.starmap(
  File "//envs/shrimpy/lib/python3.11/multiprocessing/pool.py", line 375, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "//envs/shrimpy/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
zarr.errors.BoundsCheckError: index out of bounds for dimension with length index out of bounds for dimension with length 2

[ziw-liu]

It also attempts to copy all channels instead of what is specified in the config, but that doesn't explain the BoundsCheckError because it does allocate enough C shape to write all channels:

shrimPy/mantis/cli/apply_affine.py

Lines 69 to 79 in 0df6135

	with open_ome_zarr(source_position_dirpaths[0]) as source_dataset:
	T, C, Z, Y, X = source_dataset.data.shape
	source_channel_names = source_dataset.channel_names
	source_shape_zyx = source_dataset.data.shape[-3:]
	source_voxel_size = source_dataset.scale[-3:]
	output_voxel_size = rescale_voxel_size(matrix[:3, :3], source_voxel_size)
	with open_ome_zarr(target_position_dirpaths[0]) as target_dataset:
	target_channel_names = target_dataset.channel_names
	Z_target, Y_target, X_target = target_dataset.data.shape[-3:]
	target_shape_zyx = target_dataset.data.shape[-3:]