Stereo image sample fails on Matrice 350 RTK

[uzgit]

Hello, I am trying to access the stereo images on my PSDK device over the E-port on my Matrice 350 RTK. However, the stereo vision sample does not work. The liveview sample works fine. See below.

joshua@companionboard:~/git/dji.Payload-SDK $ sudo ./build/bin/dji_sdk_demo_linux_cxx
'Logs/latest.log' -> 'DJI_0007_20231210_02-21-54.log'
[0.004][core]-[Info]-[DjiCore_Init:101) Payload SDK Version : V3.8.0-beta.0-build.1929
[0.032][adapter]-[Info]-[DjiAccessAdapter_Init:228) Identify aircraft series is Matrice 350 Series
[0.032][adapter]-[Info]-[DjiAccessAdapter_Init:261) Identify mount position type is Extension Port Type
[0.035][adapter]-[Info]-[DjiAccessAdapter_Init:362) Identity uart0 baudrate is 921600 bps
[2.095][adapter]-[Info]-[DjiPayloadNegotiate_Init:219) Waiting payload negotiate finish.
[5.176][adapter]-[Info]-[DjiPayloadNegotiate_Init:219) Waiting payload negotiate finish.
[7.837][core]-[Info]-[DjiIdentityVerify_UpdatePolicy:474) Updating dji sdk policy file...
[8.838][core]-[Info]-[DjiIdentityVerify_UpdatePolicy:482) Update dji sdk policy file successfully
[8.862][core]-[Info]-[DjiCore_Init:169) Identify AircraftType = Matrice 350 RTK, MountPosition = Extension Port, SdkAdapterType = None
[8.875][core]-[Info]-[DjiCore_ApplicationStart:226) Start dji sdk application
[8.875][user]-[Info]-[DjiUser_ApplicationStart:363) Application start.
[9.538][infor]-[Info]-[DjiAircraftInfo_NotifyMobileAppInfoHandle:689) Set mobile app info, language is English, screen type is Big Screen

| Available commands: |
| [0] Fc subscribe sample - subscribe quaternion and gps data |
| [1] Flight controller sample - you can control flying by PSDK |
| [2] Hms info manager sample - get health manger system info by language |
| [a] Gimbal manager sample - you can control gimbal by PSDK |
| [c] Camera stream view sample - display the camera video stream |
| [d] Stereo vision view sample - display the stereo image |
| [e] Run camera manager sample - you can test camera's functions interactively |
| [f] Start rtk positioning sample - you can receive rtk rtcm data when rtk signal is ok |

d
[21.049][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 3, cmdSet = 36, cmdId = 50
[21.350][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 2, cmdSet = 36, cmdId = 50
[21.651][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 1, cmdSet = 36, cmdId = 50
[21.952][linker]-[Error]-[DjiCommand_SendAsyncHandle:906) Command async send error 0
[21.952][linker]-[Error]-[DjiCommand_SendSync:489) sendSync callback timeout
[21.952][perception]-[Error]-[DjiPerception_TriggerStereoCamParamsPushing:766) Trigger perception parameters pushing error.
[21.952][perception]-[Error]-[DjiPerception_GetStereoCameraParameters:406) Trigger stereo camera parameters push failed, error code: 364688.
[21.952][user]-[Error]-[DjiUser_RunStereoVisionViewSample:131) Get camera parameters failed, return code:0x000000E1

| Available commands: |
| [0] Fc subscribe sample - subscribe quaternion and gps data |
| [1] Flight controller sample - you can control flying by PSDK |
| [2] Hms info manager sample - get health manger system info by language |
| [a] Gimbal manager sample - you can control gimbal by PSDK |
| [c] Camera stream view sample - display the camera video stream |
| [d] Stereo vision view sample - display the stereo image |
| [e] Run camera manager sample - you can test camera's functions interactively |
| [f] Start rtk positioning sample - you can receive rtk rtcm data when rtk signal is ok |

c
Please choose the stream demo you want to run

--> [0] Normal RGB image display
--> [1] Binary image display
--> [2] Faces detection demo
--> [3] Tensorflow Object detection demo

0
Please enter the type of camera stream you want to view

--> [0] Fpv Camera
--> [1] Main Camera
--> [2] Vice Camera
--> [3] Top Camera

0
Please enter the 'q' or 'Q' to quit camera stream view

Is is possible to view the stereo images on this setup? If not, is it possible to view them over the PSDK port?

[dji-dev]

Agent comment from Leon in Zendesk ticket #101636:

Dear Developer,

Hello, thank you for contacting DJI.

This is supported by M350. May I ask if you are using a BULK link? Currently only BULK links are supported to obtain this image.

Thank you for your support of DJI products.

Best Regards,
DJI Dajiang innovation SDK technical support

°°°

[uzgit]

I am using a network link, so I suppose that explains why it does not work. I will try to set up a USB bulk device instead. Is the Liveview of payload video feeds available over the USB bulk device as well?

[dji-dev]

Agent comment from Leon in Zendesk ticket #101636:

Dear Developer,

Hello, thank you for contacting DJI.

The acquisition of payload video source is supported whether it is a NETWORK link or a BULK link.

Thank you for your support of DJI products.

Best Regards,
DJI Dajiang innovation SDK technical support

°°°

[uzgit]

So I have set up the BULK link now, and I am able to run the Liveview sample successfully. However, when I try to run the stereo image sample, I get the following error:

[23.398][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 3, cmdSet = 36, cmdId = 50 
[23.699][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 2, cmdSet = 36, cmdId = 50 
[24.000][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 1, cmdSet = 36, cmdId = 50 
[24.301][linker]-[Error]-[DjiCommand_SendAsyncHandle:906) Command async send error 0 
[24.301][linker]-[Error]-[DjiCommand_SendSync:489) sendSync callback timeout 
[24.301][perception]-[Error]-[DjiPerception_TriggerStereoCamParamsPushing:766) Trigger perception parameters pushing error. 
[24.301][perception]-[Error]-[DjiPerception_GetStereoCameraParameters:406) Trigger stereo camera parameters push failed, error code: 353968. 
[24.301][user]-[Error]-[DjiUser_RunStereoVisionViewSample:131) Get camera parameters failed, return code:0x000000E1

Could you please let me know anything about this error?

Also, what is the necessary BULK configuration? In hal_usb_bulk.h it is implied that there have to be two separate BULK connections, but I currently only have one, based on the script described at the following link: here. If I try to create multiple bulk link then the startup_bulk executable fails. It is the script for the Matrice 30 so that may be why. Please provide a script for the Matrice 300/350 if you have it.

[uzgit]

I have now made two bulk devices using the following code snippet:

mkdir -p /dev/usb-ffs 

echo "Starting bulk1..."
cfg_str="${cfg_str}+BULK1" 
func=functions/ffs.bulk1
mkdir -p "${func}" 
mkdir -p /dev/usb-ffs/bulk1 
ln -sf "${func}" configs/c.1/ 
mount -o mode=0777 -o uid=2000 -o gid=2000 -t functionfs bulk1 /dev/usb-ffs/bulk1
/home/joshua/setup/startup_bulk/startup_bulk /dev/usb-ffs/bulk1 &
sleep 3
echo "Started bulk1..."

mkdir -p "${cfg}/strings/0x409" 
echo "${cfg_str:1} " > "${cfg}/strings/0x409/configuration" 

echo "Starting bulk2..."
cfg_str="${cfg_str}+BULK2" 
func=functions/ffs.bulk2
mkdir -p "${func}" 
mkdir -p /dev/usb-ffs/bulk2
ln -sf "${func}" configs/c.1/ 
mount -o mode=0777 -o uid=2000 -o gid=2000 -t functionfs bulk2 /dev/usb-ffs/bulk2
/home/joshua/setup/startup_bulk/startup_bulk /dev/usb-ffs/bulk2 &
sleep 3
echo "Started bulk2..."

mkdir -p "${cfg}/strings/0x409" 
echo "${cfg_str:1} " > "${cfg}/strings/0x409/configuration"

So this gives
/dev/usb-ffs/
├── bulk1
│   ├── ep0
│   ├── ep1
│   └── ep2
└── bulk2
├── ep0
├── ep1
└── ep2

The Liveview sample still works, but the stereo view sample gives the same error as before.

[dji-dev]

Agent comment from Leon in Zendesk ticket #101636:

Dear Developer,

Hello, thank you for contacting DJI.

Are you referring to the fact that this error is still reported?

[23.398][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 3, cmdSet = 36, cmdId = 50 [23.699][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 2, cmdSet = 36, cmdId = 50 [24.000][linker]-[Warn]-[DjiCommand_SendAsyncHandle:894) Command async send retry: index = 0, retryTimes = 1, cmdSet = 36, cmdId = 50 [24.301][linker]-[Error]-[DjiCommand_SendAsyncHandle:906) Command async send error 0 [24.301][linker]-[Error]-[DjiCommand_SendSync:489) sendSync callback timeout [24.301][perception]-[Error]-[DjiPerception_TriggerStereoCamParamsPushing:766) Trigger perception parameters pushing error. [24.301][perception]-[Error]-[DjiPerception_GetStereoCameraParameters:406) Trigger stereo camera parameters push failed, error code: 353968. [24.301][user]-[Error]-[DjiUser_RunStereoVisionViewSample:131) Get camera parameters failed, return code:0x000000E1

This error indicates that there is still a problem with the data transmission and reception of your BULK link. You can test your BULK link according to the test procedure at the end of this article to ensure that the data is passed.
https://sdk-forum.dji.net/hc/zh-cn/articles/10232604141465-%E6%A0%91%E8%8E%93%E6%B4%BE4B%E9%85%8D%E7%BD%AEUSB-device-RNDIS-%E5%92%8C-BULK

Thank you for your support of DJI products.

Best Regards,
DJI Dajiang innovation SDK technical support

°°°

[uzgit]

I made this installation script for setting up the PSDK with the Raspberry Pi 5 with the Matrice 350 via the E-port with USB bulk, and I am able to see the liveview and stereo images now. Hopefully it can help someone.

[dji-dev]

Agent comment from Leon in Zendesk ticket #101636:

Dear uzgit,

Greetings. We appreciate your contact with DJI - Da-Jiang Innovations.

Thank you for sharing. We will recommend it to all developers applicable to this platform. We believe that your script can help many developers shorten the development time of PSDK and save them the time needed to set up the environment.

Thank you for your support of DJI products! Wishing you all the best!

Best Regards,

DJI - Da-Jiang Innovations SDK Technical Supportq

°°°