VC Driver installation is always failing For Jetson Nano 4G B01 Board

[Frennz]

IMX412 on Jetson Nano 4GB B01 Installation

I'm new to the Jetson Nano Platform and got the VC IMX412 Sensor for Machine Vision Purposes.
I followed all instructions for the Recovery mode, connected it to an Ubuntu 18 LTS and 20 LTS Host, but Building / Flashing the Configuration was never Successful for me and I used a whole day trying all the different methods to install the Kernel config for the sensor without any Success.

My tegra210-camera-vc-mipi-cam.dtsi configuration is the following:

{..}

#define VC_MIPI_LANES 4 // 1, 2 Lanes (4 For the IMX412)

{..}

#if VC_MIPI_CAM_0 == 1
i2c@0 {
#if VC_MIPI_MANUFACTURER == 1
vc_mipi_cam0: vc_mipi@1a {
reg = <0x1a>;

{..}

            // ----------------------------------------------------
            // If you want to use GStreamer with nvarguscamerasrc
            // you have to adjust this settings
            physical_w              = "6.287"; 
            physical_h              = "4.712";
            // ----------------------------------------------------
            // This node is needed by the Tegra framework.
            // You don't have to change any settings if just want to use the V4L API.
            mode0 {
                num_lanes                = NUM_LANES;
                tegra_sinterface         = "serial_a";
                embedded_metadata_height = VC_MIPI_METADATA_H;
                readout_orientation      = "0";
                // ----------------------------------------------------
                // If you want to use GStreamer with nvarguscamerasrc
                // you have to adjust this settings. 
                active_l                 = "0";
                active_t                 = "0";
                active_w                 = "4032";
                active_h                 = "3040";
                pixel_t                  = "bayer_rggb";
                min_gain_val             = "0";         // mdB
                max_gain_val             = "51000";     // mdB
                step_gain_val            = "50";       // mdB
                default_gain             = "0";         // mdB
                min_exp_time             = "1";         // us
                max_exp_time             = "1000000";   // us
                step_exp_time            = "1";         // us
                default_exp_time         = "10000";     // us
                // For applications like argus_camera, there should
                // be a minimal frame rate greater than zero!
                min_framerate            = "1000";      // mHz
                max_framerate            = "40000";     // mHz
                step_framerate           = "100";       // mHz
                default_framerate        = "40000";     // mHz
                // ----------------------------------------------------
                gain_factor              = "1000";
                exposure_factor          = "1000000";
                framerate_factor         = "1000";
                inherent_gain            = "1";
                min_hdr_ratio            = "1";
                max_hdr_ratio            = "1";
                line_length              = "0";
                phy_mode                 = "DPHY";
                discontinuous_clk        = "no";
                mclk_khz                 = "24000";
                pix_clk_hz               = "800000000";
                mclk_multiplier          = "9.33";
                cil_settletime           = "0";
                dpcm_enable              = "false";

The Configuration was my first mayor confusion point because The values in the table on the README is Different from the Specifications provided from the Website: https://www.vision-components.com/fileadmin/external/documentation/hardware/VC_MIPI_Camera_Module/index.html
and is even wrong when looking in the example Configuration for the IMX226 that uses the right configuration from the Specifications and not the table values in the README.

If you look at the IMX226 example config and the values in the table, there is a mismatch.
But it's correct if looking at the IMX226 Specifications on the Website?

Now the Building errors
I Installed sudo apt install git
I Installed sudo apt install build-essential
ONCE BEFORE and ONCE NOT at all before running ./quickstart.sh
to try if it's an issue with the build tools
but errors always arose

In Ubuntu LTS 20

I Got the Errors:

1. FATAL ERROR: Unable to parse input tree

Error: /home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-camera-vc-mipi-cam.dtsi:82.38-39 syntax error
FATAL ERROR: Unable to parse input tree
make[2]: *** [/home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/Makefile:120: arch/arm64/boot/dts/ddot/ddot/ddot/ddot/ddot/ddot/hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0002-p3449-0000-b00.dtb] Error 1
DTC arch/arm64/boot/dts/ddot/ddot/ddot/ddot/ddot/ddot/hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0002-p3449-0000-a02.dtb
make[1]: *** [arch/arm64/Makefile:154: dtbs] Error 2
make: *** [Makefile:171: sub-make] Error 2

2. OK. I modified my tegra210-camera-vc-mipi-cam.dtsi Config.

No Syntax Errors anymore, but the build process stops with this feedback:

Installing QEMU binary in rootfs
mknod: /home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/rootfs/dev/random: File exists

3. OK. I Delete the build folder and Re-Build

dpkg: error processing package nvidia-l4t-gputools (--install):
installed nvidia-l4t-gputools package post-installation script subprocess returned error exit status 1

I Have no Idea how to Proceed and Switch My Host Machine to Ubuntu 18 LTS

In Ubuntu LTS 18

I Got the Errors:

1. Permission Error on /usr/(my host)/ and on the desktop

Ok I run the ./quickstart.sh with sudo privileges as : sudo ./quickstart.sh

2. Next Error : fatal: not in a git directory
   nvcommon_build.sh
   nvbuild.sh
   Initialized empty Git repository in /home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/source/public/.git/
   fatal: not in a git directory

I re-run the ./quickstart.sh Script

3. Same error : mknod: /home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/rootfs/dev/random: File exists

QEMU binary is not available, looking for QEMU from host system
Found /usr/bin/qemu-aarch64-static
Installing QEMU binary in rootfs
mknod: /home/machine/Desktop/vc_mipi_nvidia/build/Nano_32.7.1/Linux_for_Tegra/rootfs/dev/random: File exists

4. OK. I Delete the build folder and Re-Build

Errors were encountered while processing:
nvidia-l4t-gputools

I Have no Idea how to Proceed

Alternative install

I looked at the instructions for integrating the driver in your own BSP. I do not know how to find the exact BSP Information what PATCH files are and HOW to apply them to Build The Kernel, but that's probably because I'm new to the Jetson system.

I Have no idea how to troubleshoot these errors

I do not know what is causing the build flash process to not succeed
If there is any possibility for help and a guided process I would be grateful for any support in that matter in English or German, my email is Franz@hnva.de

Appended is my tegra210-camera-vc-mipi-cam.dtsi
Named : tegra210-camera-vc-mipi-cam_IMX412CFG

tegra210-camera-vc-mipi-cam_IMX412CFG.zip

[bazo80]

Hello @Frennz,

the Jetson Nano SystemOnModule is capable of using 4 lanes, but the DevKit exposes only 2 of them. That means

#define VC_MIPI_LANES 2

must be set. Otherwise the build process of the device tree will return an error.
With that lane setting, the IMX412 can only use

max_framerate            = "20000";     // mHz
default_framerate        = "20000";     // mHz

The Readme.md still has an error regarding the example dtsi. The rggb info is swapped for the IMX226. That will be fixed in the next release.
For configuring the sensor, you should use the values from the foldable bullet points from the Readme.md with respect to the lane number. There are some small differences between the vc website and the table in the Readme. The vc website states the whole sensor array and it's measurements in mm. The table in the Readme uses the measurements according to the used sensor array.
So the physical_w should be 6.250 (Readme bulletpoint) instead of 6.287 (vc website). The size measurements [mm] affects the nvarguscamerasrc only, but should not cause an error during compilation.

---

The build-essentials package should be installed by the setup.sh script (bin folder) and the setup.sh script is called by quickstart.sh script. This happens automatically when calling quickstart. When using a freshly fetched repo, there is no build directory with Linux_for_Tegra/rootfs/dev/random...
So, I'm wondering where this message comes from, because at setup time the function install_system_tools is invoked before the extraction/build of the sources. Did you installed the build essentials manually with some parameters?

---

The quickstart.sh script must not be run as root! When there is a root operation necessary, you will be prompted to the root password. The quickstart script is calling the three sub scripts setup.sh, build.sh and flash.sh. The quickstart script is demanding the recovery mode at the beginning of the procedure, but the recovery mode is necessary for the flashing procedure as the last part.

We can also go through the process step by step.
You have got a freshly cloned vc_mipi repo. In the bin directory, call:

./setup.sh -o

For the first try, you can leave out the configuration of the sensor. You can configure it after the process.
The setup routine should end with applying patches and copying the dtsi files. If an error occurs, please upload the error message. If it succeeds, call:

./build.sh -a

If this script terminates with a failure, please upload the exact message. Otherwise it has built the kernel and you can flash your device.
Therefore, you should get into the recovery mode. You can check this by calling

lsusb

If the message:
NVIDIA Corp. APX
occurs in the list, the device can be flashed by:

./flash.sh -a

This will take some time. After the device has boot up the first time, you can setup your target system or shut it down and put it in the recovery mode again.

By calling

./setup.sh -c

the sensor can be configured.

./build.sh -d

will build the device tree only.

./flash.sh -d

will flash the device tree only.

If there is an error at some point, you can attach the log.

Regards

[Frennz]

Hello @Frennz,

the Jetson Nano SystemOnModule is capable of using 4 lanes, but the DevKit exposes only 2 of them. That means

#define VC_MIPI_LANES 2

must be set. Otherwise the build process of the device tree will return an error.

With that lane setting, the IMX412 can only use

max_framerate            = "20000";     // mHz

default_framerate        = "20000";     // mHz

The Readme.md still has an error regarding the example dtsi. The rggb info is swapped for the IMX226. That will be fixed in the next release.

For configuring the sensor, you should use the values from the foldable bullet points from the Readme.md with respect to the lane number. There are some small differences between the vc website and the table in the Readme. The vc website states the whole sensor array and it's measurements in mm. The table in the Readme uses the measurements according to the used sensor array.

So the physical_w should be 6.250 (Readme bulletpoint) instead of 6.287 (vc website). The size measurements [mm] affects the nvarguscamerasrc only, but should not cause an error during compilation.

---

The build-essentials package should be installed by the setup.sh script (bin folder) and the setup.sh script is called by quickstart.sh script. This happens automatically when calling quickstart. When using a freshly fetched repo, there is no build directory with Linux_for_Tegra/rootfs/dev/random...

So, I'm wondering where this message comes from, because at setup time the function install_system_tools is invoked before the extraction/build of the sources. Did you installed the build essentials manually with some parameters?

---

The quickstart.sh script must not be run as root! When there is a root operation necessary, you will be prompted to the root password. The quickstart script is calling the three sub scripts setup.sh, build.sh and flash.sh. The quickstart script is demanding the recovery mode at the beginning of the procedure, but the recovery mode is necessary for the flashing procedure as the last part.

We can also go through the process step by step.

You have got a freshly cloned vc_mipi repo. In the bin directory, call:

./setup.sh -o

For the first try, you can leave out the configuration of the sensor. You can configure it after the process.

The setup routine should end with applying patches and copying the dtsi files. If an error occurs, please upload the error message. If it succeeds, call:

./build.sh -a

If this script terminates with a failure, please upload the exact message. Otherwise it has built the kernel and you can flash your device.

Therefore, you should get into the recovery mode. You can check this by calling

lsusb

If the message:

NVIDIA Corp. APX

occurs in the list, the device can be flashed by:

./flash.sh -a

This will take some time. After the device has boot up the first time, you can setup your target system or shut it down and put it in the recovery mode again.

By calling

./setup.sh -c

the sensor can be configured.

./build.sh -d

will build the device tree only.

./flash.sh -d

will flash the device tree only.

If there is an error at some point, you can attach the log.

Regards

Thank you for the explanation I got it running after modifying the QuickStart script for the configuration and then running the build and flash command manually. I changed the lane to 2 lanes but left all the values the same and it somehow passed the v4l2 test and I ran gst-launch-1.0 nvarguscamerasrc ! nvoverlaysink without any problems, it even reported 40fps. I'm not sure if this is just reading the values from the device tree or if it's dropping frames or something I will follow this up later. I need at least 24 frames for my project and since the limitation for full sensor readout is 20fps with 2 lanes I'd like to ask how I should modify my device tree for a config that runs at least to 24 fps, if you can help me with that I would be grateful.

[bazo80]

Hello @Frennz,

the gstreamer/nvarguscamerasrc has an option text-overlay=true and also -v
Then you should see the dropping rate.

For your fps problem there are several solutions. You can either relinquish some lines of the sensor array. The IMX412 is able to get faster by leaving out horizontal lines. Please have a look at the https://github.com/VC-MIPI-modules/vc_mipi_nvidia/blob/master/doc/ROI_CROPPING.md document
There is also the possibility of the binning mode, which is still in development or you can purchase an Auvidea JNX30 carrier board, which makes use of 4 lanes.

Regards

[bazo80]

Ok, I see, we have to correct the entry "frame rate increase" from no to yes and also modify the vc website, because now the sensor is able to increase it's frame rate by leaving out lines.

[Frennz]

Ok, I see, we have to correct the entry "frame rate increase" from no to yes and also modify the vc website, because now the sensor is able to increase it's frame rate by leaving out lines.

That is awesome to hear thank you for the in depth support and the fast responses. Binning is also marked as not possible on the VC website if that's also possible for super sampling that would be great. Last question ... In this Datasheet from Sony: https://www.sony-semicon.com/files/62/flyer_security/IMX412-AACK_Flyer.pdf the framerate for 10 bit is 60fps and 40 fps for 12bit for the IMX412-aack. I'm not sure what the difference is to the IMX412-aack-c or the IMX-412-aack-d and if this also applies to your vc module or if it's not possible because data limitations of the the csi2/mipi interface?

[bazo80]

In reference to -c and -d I'm stumped.

I am about to integrate the binning feature, but I'm not sure when it will be ready.

[Frennz]

In reference to -c and -d I'm stumped.

I am about to integrate the binning feature, but I'm not sure when it will be ready.

-C is color and -D is Monochrome but vc only makes the IMX412 in the Color Version

Can you verify the Frame rate over 4 Lanes?
I got the JN30A 38346-3 Board as you recommended. Can I now use the 4 Lanes in the Device tree under "#define VC_MIPI_LANES [2 or 4] // 1, 2 Lanes" ?.

Also, this Data Sheet says the Sensor can deliver 60 frames/s https://www.sony-semicon.com/files/62/flyer_security/IMX412-AACK_Flyer.pdf at full sensor read out is it possible with your Mipi Board?