Colour error/corruption when viewing decompressed DICOM images

[SamJK001]

In a similar vein to #1577, I am experiencing some trouble viewing ultrasound images once I have decompressed them.

Here is my the code I am using:

import os
from deid.dicom import DicomCleaner
from pydicom import dcmread
from pydicom.pixel_data_handlers import convert_color_space
import pydicom
import PIL
import numpy
# import gdcm

src_dir = "~/Desktop/Test_Files/Temp/"
dst_dir = "~/Desktop/Test_Files/Cleaned/"
EV_files = os.listdir(src_dir)

client = DicomCleaner()

compression_list = []

for file in EV_files:
    file_path = src_dir + file
    dicom_list = os.listdir(file_path)
    print("Cleaning " + file)
    
    for d in dicom_list:
        dst_folder = dst_dir + file
        client = DicomCleaner(output_folder=dst_folder)
        f_path = file_path + "/" + d
        ds = dcmread(f_path)
        if ds.file_meta.TransferSyntaxUID.is_compressed is True:
            print("Decompressing " + d)
            rgb = convert_color_space(ds.pixel_array, "YBR_FULL", "RGB")
            ds.decompress('pillow')
            dcm_decomp = f_path + "decomp"
            ds.save_as(dcm_decomp)
            client.detect(dcm_decomp)
            client.clean()
            client.save_dicom()

        else:
            client.detect(f_path)
            client.clean()
            client.save_dicom()

I'm expecting an image like this:

Instead, it produces an image like this:

I tried @scaramallion's suggestion from #1577 using convert_color_space(), yet this had no effect.

Any suggestions would be greatly appreciated! Thanks in advance :)

[scaramallion]

That's interesting, could you attach an anonymised version of the dataset (or even just one with the group 0028 elements ds.group_dataset(0x0028) and the pixel data)?

I'd say there's an incorrect image pixel module element, if I had to guess, since the image appears jumbled rather than just in the incorrect color space.

[SamJK001]

Here is a trial script just to test the decompression:

from pydicom import dcmread
import gdcm

path = "D:/Test_Files/Temp/8899/US000309"

ds = dcmread(path)

print(ds.file_meta.TransferSyntaxUID)
print(ds.PhotometricInterpretation)

ds.decompress("gdcm")

print(ds.file_meta.TransferSyntaxUID)
ds.PhotometricInterpretation = "RGB"
print(ds.PhotometricInterpretation)

ds.save_as("D:/Test_Files/Temp/8999/test.dcm")

I get the following error:

Traceback (most recent call last):
  File "d:\Scripts\matlabplot_test.py", line 19, in <module>
    ds.save_as("D:/Test_Files/Temp/8999/test.dcm")
packages\pydicom\dataset.py", line 1887, in save_as
    pydicom.dcmwrite(filename, self, write_like_original)
  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pydicom\filewriter.py", line 1007, in dcmwrite
    dataset.PixelData = dataset.pixel_array.tobytes()
  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pydicom\dataset.py", line 1716, in pixel_array
    self.convert_pixel_data()
    self._convert_pixel_data_without_handler()
  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pydicom\dataset.py", line 1545, in _convert_pixel_data_without_handler
    raise last_exception
  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pydicom\dataset.py", line 1525, in _convert_pixel_data_without_handler
    self._do_pixel_data_conversion(handler)
  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-  File "C:\Users\skrasner\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pydicom\pixel_data_handlers\numpy_handler.py", line 312, in get_pixeldata
    raise ValueError(
ValueError: The length of the pixel data in the dataset (5663576 bytes) doesn't match the expected length (125280000 bytes). The dataset may 
be corrupted or there may be an issue with the pixel data handler.

Not sure where I am going wrong :(

[scaramallion]

It seems like you have compressed Pixel Data but an uncompressed transfer syntax

[SamJK001]

The Transfer Syntax was initially 1.2.840.10008.1.2.4.50 before running the decompression and was converted to 1.2.840.10008.1.2.1.

Is there a way to decompress the Pixel Data? Is this a common format of dicoms or could it be due to machine configurations/export method?

[scaramallion]

Looking at the source code, I don't think decompress() sets the Pixel Data using the decompressed array. Try:

from pydicom import dcmread
from pydicom.pixel_data_handlers import convert_color_space
from pydicom.uid import ExplicitVRLittleEndian

ds = dcmread(...)
if ds.file_meta.TransferSyntaxUID.is_compressed:
    arr = ds.pixel_array
    if ds.PhotometricInterpretation in ("YBR_FULL", "YBR_FULL_422"):
        arr = convert_color_space(arr, ds.PhotometricInterpretation, "RGB")
        ds.PhotometricInterpretation = "RGB"
    else:
        raise NotImplementedError(
            f"Script doesn't handle a Photometric Interpretation of '{ds.PhotometricInterpretation}'"
        )

    ds.PixelData = arr.tobytes()
    ds.file_meta.TransferSyntaxUID = ExplicitVRLittleEndian
    ds.save_as("out.dcm")

[SamJK001]

You're a legend! That has solved the compression issues completely! Thank you very much.