Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After reducing the size of the large image to the image size (752 * 480) of the Euroc dataset, the output result became worse #449

Open
SerpentWindy opened this issue Apr 29, 2024 · 4 comments

Comments

@SerpentWindy
Copy link

SerpentWindy commented Apr 29, 2024

My image size is 1164 * 874, and the image includes the front hood of the car. During the one kilometer journey, VINS did not experience a system reboot, but the trajectory distance and angle were inaccurate. Then I resized and cropped the images, and ran VINS again using the modified images as the dataset. However, there was a "big translation" system reboot error during the run. I want to know why the result is worse after resizing and cutting ?
The reason why I resized and collapsed the image is as follows:

  1. I think the front hatch will affect the effectiveness of feature tracking and visual modeling, so I tried to crop the area of the front hatch.
  2. I think the feature detection parameters I use are all parameters corresponding to the Euroc dataset, and my image size is too large compared to (752,480). These feature detection parameters are not suitable for my image, so I want to resize the image with a size of 1164 * 874 to (752,480).

In summary, I resized and collapsed the 1164 * 874 image to obtain a 752 * 480 size image, with the front hatch removed. Because I changed the image size, I needed to modify the camera intrinsic matrix. Without changing other parameters, I encountered a "big translation" system reboot error while running VINS,

My modification process is as follows:

1.   I resized the image with dimensions of 1164 * 874 to (752,600),
2.   crop the (752,600) size image to (752,480) size,
3.   a new bag was created using  (752,480) images and IMU data,
4.   modify the camera's intrinsic matrix based on my resizing and cropping.

My original fx=910.0, fy=910.0, cx=582.0, cy=437.0,
After resize, fx=910.0752/1164=587.90378, fy=910.0600/874=624.71395881, cx=582.0752/1164=376.0, cy=437.0600/874=300,
After crop, fx=587.90378, fy=624.71395881, cx=376.0, cy=300-(600-480)/2=240.0

@SerpentWindy
Copy link
Author

This is my original image
1713499064110488064

@SerpentWindy
Copy link
Author

This is my modified image
1713499064110488064 - 复件(2)

@SerpentWindy
Copy link
Author

SerpentWindy commented Apr 29, 2024

This is my original parameter table:

#camera calibration
model_type: PINHOLE
camera_name: camera

image_width: 1164
image_height: 874
distortion_parameters:
k1: 0.1322964747
k2: -0.2047703774
p1: 0.0000790454
p2: 0.00001595080
projection_parameters:
fx: 9.1e+02
fy: 9.1e+02
cx: 5.82e+02
cy: 4.37e+02

#Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 1 #0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
2 Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [0.0017453, -0.0017423, 0.9999970,
0.9999970, 0.0017484, -0.0017423,
-0.0017453, 0.9999970, 0.0017453]
#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [0.01,0.01,0.01]

#feature traker paprameters
max_cnt: 150 # max feature number in feature tracking
min_dist: 30 # min distance between two features
freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image
F_threshold: 1.0 # ransac threshold (pixel)
show_track: 1 # publish tracking image as topic
equalize: 1 # if image is too dark or light, trun on equalize to find enough features
fisheye: 0 # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time
max_num_iterations: 8 # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance
acc_n: 0.08 # accelerometer measurement noise standard deviation. #0.2 0.04
gyr_n: 0.004 # gyroscope measurement noise standard deviation. #0.05 0.004
acc_w: 0.00004 # accelerometer bias random work noise standard deviation. #0.02
gyr_w: 2.0e-6 # gyroscope bias random work noise standard deviation. #4.0e-5
g_norm: 9.81007 # gravity magnitude

#loop closure parameters
loop_closure: 1 # start loop closure
load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path'
fast_relocalization: 0 # useful in real-time and large project
pose_graph_save_path: "/home/nvidia/vio_output/pose_graph/" # save and load path

#unsynchronization parameters
estimate_td: 0 # online estimate time offset between camera and imu
td: 0.0 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters
rolling_shutter: 1 # 0: global shutter camera, 1: rolling shutter camera
rolling_shutter_tr: 0.029 # unit: s. rolling shutter read out time per frame (from data sheet).

#visualization parameters
save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0
visualize_imu_forward: 0 # output imu forward propogation to achieve low latency and high frequence results
visualize_camera_size: 0.4 # size of camera marker in RVIZ



@SerpentWindy
Copy link
Author

SerpentWindy commented Apr 29, 2024

This is my modified parameter table:

#camera calibration
model_type: PINHOLE
camera_name: camera

image_width: 752
image_height: 480
#distortion_parameters:
k1: 0.1322964747
k2: -0.2047703774
p1: 0.0000790454
p2: 0.00001595080
#projection_parameters:
fx: 587.90378
fy: 624.71395881
cx: 376.0
cy: 240.0

#Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 1 #0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
2 Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [0.0017453, -0.0017423, 0.9999970,
0.9999970, 0.0017484, -0.0017423,
-0.0017453, 0.9999970, 0.0017453]
#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [0.01,0.01,0.01]

#feature traker paprameters
max_cnt: 150 # max feature number in feature tracking
min_dist: 30 # min distance between two features
freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image
F_threshold: 1.0 # ransac threshold (pixel)
show_track: 1 # publish tracking image as topic
equalize: 1 # if image is too dark or light, trun on equalize to find enough features
fisheye: 0 # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time
max_num_iterations: 8 # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance
acc_n: 0.08 # accelerometer measurement noise standard deviation. #0.2 0.04
gyr_n: 0.004 # gyroscope measurement noise standard deviation. #0.05 0.004
acc_w: 0.00004 # accelerometer bias random work noise standard deviation. #0.02
gyr_w: 2.0e-6 # gyroscope bias random work noise standard deviation. #4.0e-5
g_norm: 9.81007 # gravity magnitude

#loop closure parameters
loop_closure: 1 # start loop closure
load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path'
fast_relocalization: 0 # useful in real-time and large project
pose_graph_save_path: "/home/nvidia/vio_output/pose_graph/" # save and load path

#unsynchronization parameters
estimate_td: 0 # online estimate time offset between camera and imu
td: 0.0 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters
rolling_shutter: 1 # 0: global shutter camera, 1: rolling shutter camera
rolling_shutter_tr: 0.029 # unit: s. rolling shutter read out time per frame (from data sheet).

#visualization parameters
save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0
visualize_imu_forward: 0 # output imu forward propogation to achieve low latency and high frequence results
visualize_camera_size: 0.4 # size of camera marker in RVIZ



Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant