Can't set "gray", "y10" or "y12" pixel formats in device tree modes if LINUX_VERSION >= 500

[taldhous-imetrum]

In order to have a choice of image formats for a connected camera in V4L2, I have found that there have to be multiple modes in the device tree - one for each image format supported by the camera. The formats included in the modes are listed by the command

v4l2-ctl --list-formats-ext

Is this expected behaviour?

In the mode section of the device tree, before Linux V500, the field pixel_t was used to specify the image format. After Linux V500 this was split into three fields, "mode_type", "pixel_phase" and "csi_pixel_bit_depth".

#if LINUX_VERSION < 500
        pixel_t                  = "bayer_rggb";
#else
        mode_type                = "bayer";
        pixel_phase              = "rggb";
        csi_pixel_bit_depth      = "10";
#endif

It is not possible to specify the pixel formats "gray", "y10" or "y12" using the new fields "mode_type", "pixel_phase" and "csi_pixel_bit_depth" because in the code*, they are combined into a pixel format string that contains an underscore character. (The above example would be "bayer_rggb10".) The field "csi_pixel_bit_depth" is required by other code so must be present.

The code will prefer to read the "pixel_t" if it is present, so a work around seems to be to include both the "pixel_t" fields
and the other fields:

        pixel_t                  = "gray";
#if LINUX_VERSION >= 500
        mode_type                = "";
        pixel_phase              = "gray";
        csi_pixel_bit_depth      = "8";
#endif

• see Linux_for_Tegra/source/public/kernel/nvidia/drivers/media/platform/tegra/camera/sensor_common.c line 382:
  ret = sprintf(pix_format, "%s_%s%d", mode_str, phase_str, depth);

[bazo80]

Hello @taldhous-imetrum,

you are right, the multiple modes are necessary for the newer v4l2 versions in order to switch the formats. The former v4l2 versions could set different formats without having multiple device-tree modes. Your proposal of taking the pixel_t into the jetpack 5 versions sounds reasonable. It would simplify the dtsi at this point. But on the other hand, if the pixel_t variable will be discontinued by nvidia one day, there must be an additional patch in order to ensure the functionalty in future. We would like to discuss your proposal internally. Thank you :)

[taldhous-imetrum]

On NVIDIA L4T 32.7.3 with my modified device tree with multiple modes, the system registers the IMX250 camera correctly, and v4l2-ctl lists available formats:

vc@XavierNX:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'GREY'
        Name        : 8-bit Greyscale
                Size: Discrete 2432x2048
                        Interval: Discrete infs (0.000 fps)

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'Y10 '
        Name        : 10-bit Greyscale
                Size: Discrete 2432x2048
                        Interval: Discrete infs (0.000 fps)

        Index       : 2
        Type        : Video Capture
        Pixel Format: 'Y12 '
        Name        : 12-bit Greyscale
                Size: Discrete 2432x2048
                        Interval: Discrete infs (0.000 fps)

and is able to grab a valid image:

vc@XavierNX:~/scripts$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2432,height=2048,pixelformat=GREY --set-ctrl bypass_mode=0,exposure=10000,trigger_mode=0 --stream-mmap --stream-count=1 --stream-to=imx250_0.raw
<

On NVIDIA L4T 35.3.1, everything behaves as above, but the saved image is corrupt and cannot be used.

[bazo80]

Did you speed up the clock settings?

sudo ./max_speed.sh

This script is residing in the target folder of the project and should be copied to your device (necessary for L4T35.x versions).

[taldhous-imetrum]

Perfect! I had not run that script. Doing so fixes my problem on L4T 35.3.1.

To satisfy my curiosity, why is this script needed on L4T 35.x but not earlier versions?

Many thanks!

[bazo80]

There is a bunch of threads in the nvidia board dealing with this topic.
E.g. https://forums.developer.nvidia.com/t/jetpack-5-0-2-camera-driver-port-leads-to-tegra-capture-vi-timeout/229259
But I could not find a thread where it was explained in a greater detail.