Bundle-adjustment-in-the-large example optimization fails for larger problems

[mariusud]

Describe the bug
Hi! Running bundle-adjustment-in-the large example runs fine on the first 3 files from Trafalgar dataset, but fails when running on larger ones.

To Reproduce
Steps to reproduce the behavior, e.g.:

1. Compile symforce examples and download dataset using download_dataset
2. Run bundle-adjustment-in-the-large on problem-126 or any of the larger files

Expected behavior
The optimization to converge and return success, as it does for the smaller sized problems

Screenshots

Environment (please complete the following information):

• Ubuntu 22.04.2 LTS
• Python 3.10.12
• SymForce Version 42c4df7

[rbrott]

The optimization goes better if you exclude the camera at index 72 for problems 126 and later (same deal with the camera at index 238 for problem 257). Camera 72 in particular seems to be poorly initialized with a position far away from the other cameras. Arguably the SymForce formulation could be made robust to this kind of outlier, though it's easy to identify and filter out prior to optimization.