image-sets-normalization pipeline

[PaulHax]

Prior art:

• https://github.com/Kitware/VolView/blob/main/src/io/itk-dicom/dicom.cpp#L174
• https://github.com/malaterre/GDCM/blob/master/Examples/Cxx/DiscriminateVolume.cxx
• https://github.com/malaterre/GDCM/blob/master/Utilities/VTK/vtkGDCMImageReader.cxx

Image sets JSON shape:
https://docs.aws.amazon.com/healthimaging/latest/devguide/understanding-image-sets.html

TODO

• Instead of Filename key, do ImageFrames
• Pull Patient, study, series tags: https://docs.aws.amazon.com/healthimaging/latest/devguide/reference-dicom-support.html
• Group files by "Patient, Study + Series + orientation + frame of reference UID, pixel size" and put all Volumes under Series with unique IDs.
• Pass our volume grouped series to Spatial Sorter.
• Return arrays of numbers as array of floats, not strings. https://github.com/malaterre/GDCM/blob/master/Source/MediaStorageAndFileFormat/gdcmJSON.cxx
• params for grouping image sets and volumes
• Check consistence of GDCM Keyword AWS Image Set property -> provide Keyword dictionary if needed
• Fix extra space at end of string tags.
• Use SpecificCharacterSet tag and DICOMTagReader::CharStringToUTF8Converter.

---

• Fix exception related tag memory management with the set.
• Support proviing just binary file headers and add test
• Optimize: Only read tags once.
• Delete gdcmDiscriminateVolume.h if still unused (may be helpful for instance sorting step)

Current output with 2 image series/sets:
actual.json.tar.gz

[thewtex]

• https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/IO/GDCM/src/itkGDCMImageIO.cxx
• gdcmImageHelper -> sorting classes
• https://github.com/malaterre/GDCM/blob/424e5f22db27ba2c7b8e8dde2020635e48b56e8d/Examples/Cxx/SortImage.cxx#L41
• https://github.com/malaterre/GDCM/blob/424e5f22db27ba2c7b8e8dde2020635e48b56e8d/Testing/Source/MediaStorageAndFileFormat/Cxx/TestIPPSorter2.cxx#L32

[thewtex]

Notes:

• We do not want to read from directories
• We want to take as inputs the dicom file headers, itk::wasm::BinaryFile or itk::wasm::BinaryStream
• In the C++ tests, these will be the full dicom binary files
• In JavaScript tests, these can just be the binary file headers, code snippet from @floryst 👍

 function* parseDicomUpToPixelData() {
  const reader = new StreamingByteReader();

  // preamble
  yield* reader.seek(128);

  // prefix
  const prefix = yield* reader.readAscii(4);
  if (prefix !== "DICM") {
    throw new Error("Not DICOM");
  }

  const info = yield* readFieldMetaInfo(reader);
  const explicitVr = info.transferSyntaxUid !== ImplicitTransferSyntaxUID;
  const littleEndian = info.transferSyntaxUid !== ExplicitVRBigEndianUID;

  yield* skipDataElementsUntil(
    reader,
    // (7fe0,0010): Pixel Data
    (group, element) => group === 0x7fe0 && element === 0x0010,
    { explicitVr, littleEndian }
  );
}

[PaulHax]

current JSON output
image-sets.json

AWS Image Set example
image-sets.json