Intrinsic matrix in .r3d recordings

[tszolar]

Hi, is there a specific reason that in the internal recording format (.r3d), there is only one intrinsic matrix stored per recording and not for each frame? Are those always values from the first frame?

I am doing some 3D reconstruction experiments with iPhone 15 Pro LIDAR where I need accurate intrinsic parameters for each frame and one intrinsic matrix per whole recording doesn't seem to be sufficient. I did also some recordings via the USB streaming mode, where the intrinsics are provided for each frame, and the principal point seems to be moving around quite a lot (presumably because of active image stabilization) during the recording.

Here is a visualization of the principal point positions for all frames in 4 short USB streaming mode recordings. In 2 recordings the phone is static and in the other 2 it is hand-held and moving. It seems to be only changing significantly in case the camera is also moving, which would further confirm the image stabilization hypothesis. Auto-focus option does not seem to have much impact on it.

[marek-simonik]

Hi,

I am sorry for the late reply. As you correctly said, the K key in the metadata file stores (some of the) coefficients of the intrinsic matrix of the first frame of the 3D video.

Back when I first released Record3D, every frame had the same intrinsic matrix, which is why I opted for storing just a single instance of intrinsic camera coefficients.

As this assumption is no longer valid, I will introduce a new key into the metadata file, which will store per-frame intrinsic coefficients. This will be available in the next Record3D update.

Thanks for suggesting this.

[marek-simonik]

This feature has been implemented in Record3D 1.10, please update.

Beginning with Record3D 1.10, the metadata file of a 3D video contains a new field called perFrameIntrinsicCoeffs, which holds per-frame intrinsic camera coefficients.