Add Bundle Adjustment Module

[wang-chen]

🚀 The feature, motivation and pitch

As stated in the title, a differential bundle adjustment module is required in many SLAM systems.
This module may require the following implemented functionalities.

• LM optimizer
• Model Jacobian Computation

Alternatives

No response

Additional context

No response

[lizimo061]

Hi there, I wrote a small example based on BAL dataset, but I found it is extremely slow even on the smallest dataset(with the vectorize option off, CUDA out of memory reported if on), any good suggestion?

class BAGraph(nn.Module):

    def __init__(self, cameras, points, intrinsics):
        super().__init__()
        self.cameras = pp.Parameter(cameras)
        self.points = nn.Parameter(points)
        self.intrinsics = nn.Parameter(intrinsics)

    def forward(self, observations, camera_index, pt_index):
        pt_cam = self.cameras[camera_index, ...].Act(self.points[pt_index, ...])
        pt_image = -pt_cam[..., :2] / torch.unsqueeze(pt_cam[..., 2], 1)
        pt_norm = torch.linalg.vector_norm(pt_image, ord=2, dim=1)
        distortion_factors = 1 + self.intrinsics[camera_index, ..., 1] * pt_norm**2 + self.intrinsics[camera_index, ...,
                                                                                                      2] * pt_norm**4
        pt_pixel = torch.unsqueeze(self.intrinsics[camera_index, ..., 0], 1) * torch.unsqueeze(distortion_factors,
                                                                                               1) * pt_image
        error = pt_pixel - observations
        return error

[wang-chen]

Hi, we are currently implementing sparse tensors, which will solve this problem. @zitongzhan

[zitongzhan]

BA is dependent on this issue: #207

[hxu296]

Hi there, I wrote a small example based on BAL dataset, but I found it is extremely slow even on the smallest dataset(with the vectorize option off, CUDA out of memory reported if on), any good suggestion?

class BAGraph(nn.Module):

    def __init__(self, cameras, points, intrinsics):
        super().__init__()
        self.cameras = pp.Parameter(cameras)
        self.points = nn.Parameter(points)
        self.intrinsics = nn.Parameter(intrinsics)

    def forward(self, observations, camera_index, pt_index):
        pt_cam = self.cameras[camera_index, ...].Act(self.points[pt_index, ...])
        pt_image = -pt_cam[..., :2] / torch.unsqueeze(pt_cam[..., 2], 1)
        pt_norm = torch.linalg.vector_norm(pt_image, ord=2, dim=1)
        distortion_factors = 1 + self.intrinsics[camera_index, ..., 1] * pt_norm**2 + self.intrinsics[camera_index, ...,
                                                                                                      2] * pt_norm**4
        pt_pixel = torch.unsqueeze(self.intrinsics[camera_index, ..., 0], 1) * torch.unsqueeze(distortion_factors,
                                                                                               1) * pt_image
        error = pt_pixel - observations
        return error

Hello, you can try an unofficial pypose-native BAL colab example here. The smallest problem (problem-49-7776-pre) works fine on both CPU and GPU, and you can change the problem by using CLI parameters.