Translation strongly affects rotation.

[HMRMike]

Hello!
First of all, I'm very impressed with the new Accela and PT. Overall the accuracy in RC13P2 is awesome and response time feels very natural and comfortable, with PS3 eye at 640X480@75FPS. The new threshold setting doesn't seem to make much difference than before, all is still very good. Thanks a lot for all this work, a perfect alternative to the old FreeTrack and obvious answer to commercial trackers that cost a fortune.

So far I have only found a single issue with my setup. When translating in the X direction, the tracker responds with very much yaw and apparently correct translation. For example in a game, the view yaws to look roughly at the original center point. If I move left, the view will move left correctly, but also will yaw to the right, giving the feeling as if I'm moving normal to a sphere. Sure I checked the output remap, it's all fine there.
My camera is located on top of my screen on the right side, about 15cm above the center LED in the clip and directly in front of the clip, as it is mounted on my right side of the headset. So the camera looks straight ahead and slightly downwards to see the LEDs exactly in the center of the frame.
Using the camera offset settings, the issue can only be resolved for one direction, and using the center command will make it as if I never set any camera offset values at all.

So I'm quite lost here. Where to begin solving this problem?

[sthalik]

Hey,

PS3 Eye has fisheye-distorting lens compared to, say, Logitech cameras. We could fix it by supplying distortion parameters.

[HMRMike]

Great!
An interesting temporary solution: Setting a smaller FOV value in PT, than the actual lens FOV, almost completely eliminates the yaw response during translation.
For example, with the lens at 75° and PT set to 75°, and use the center command to zero Z, I get almost the same yaw as X in "game value" box. Same with the lens at 56° and PT set to 56°.
Now, If I enter 45° in PT, again with zeroing, with the lens at 75° the yaw response becomes +-5, while the X translation is correct at +-35.
At lower values like 30 and 20, the yaw response again grows too much to +-20.

[sthalik]

Calibrating a camera is hard. Every time I tried, it either didn't help or made it worse. Any of you guys know how to use OpenCV calibration procedure for an arbitrary webcam?

[sthalik]

@HMRMike can you confirm "optimal" skewed FOV values for 56 and 75 PS3 Eye lens modes?

Do they still work if you move forward or backward the model?

[sthalik]

@HMRMike I'd like to know if you can now use Accela without deadzone and smoothing. What's your Accela config?

[HMRMike]

Finding the optimal FOV setting turned out to be very easy. Move in X to the edge of the frame, then adjust FOV until yaw reaction comes close to zero. Move to the other side to check. Tested with a good office chair instead of a wobbly moving human! Nicely kept parallel to the sensor plane, so the only motion was in the X axis. Optimal final yaw response was considered within +-5 in game data.

Tested two cameras, an old TEAC MX-6 with an unknown lens, measured 24° FOV (probably not the original lens) and the PS3 Eye.
Instead of detailing results here's the conclusion: In all three cases the optimal FOV value was about 1.6 times smaller than the real camera FOV. I hope this number is useful.
And yes, after moving in Z, yaw response was still minimal.

As for accela settings, I can't use it without any filtering. My PS3 cam still has it's IR filter, so I must use slightly higher gain settings, making the image slightly noisy. Also, at 75FPS, the camera itself gives a rather noisy image, even at minimal gain, with lots of vertical stripes.
Camera at 640X480@75FPS. Using yaw and pitch mapping set to 90 input and 180 output.
Accella settings:
Smoothing 10ms
Rotation sensitivity 0.6°
deadzone 0
Translation sensitivity 0.5mm (filters out all translation "noise")
deadzone 0mm
This leaves a slight "floating" movement of a couple pixels when I don't move (but in fact I do, we all breathe and don't keep our head perfectly still), very slow and subtle. Feels natural.

[sthalik]

The current focal length equation we're using is confirmed by multiple sources

http://answers.opencv.org/question/1149/focal-length-and-calibration/
http://www.cinematography.com/index.php?showtopic=5207

and the list goes on.

Your ratio of 1.6 looks attractive since:

>>> tan(56 / 1.6 / 180 * pi) / tan(56 / 180. * pi) == 0.47229594807971803

It's about half.

I've confirmed with my Aruco marker however that setting one FOV value to unbreak yaw, doesn't fix it for all Z values. Move the marker backward or forward and the yaw value changes. We can also observe the fisheye distortion on the screen with Z changing, by how "rotated" marker looks like when moved.

[sthalik]

I have something else though, from this link http://answers.unity3d.com/questions/588295/how-convert-camera-fov-to-diagonal-fov.html

Maybe the specified PS3 Eye camera FOV is diagonal, and this equation is needed.

>>> 2.0*atan(tan(56*pi/180./2.0)/sqrt(1 + ((800./640.) ** 2)))*180/pi === 36.74845924107028

Try using this value and see if it works for you. I've tested on user-supplied video with translation but the videos aren't exhaustive enough to check.

[sthalik]

FWIW this also has relevant stuff mrdoob/three.js#1239

[HMRMike]

36.74845 is close enough to my "optimal" found value (~35) and doesn't show much difference in output.

The PS3 lens seems to have it's FOV specified in diagonal, after all.
This seems correct when assuming the lens is rectilinear (we can see it really doesn't distort much at all) and measurements using photos. But it does have a slight barrel distortion, making it somewhere between a true rectilinear lens and a fisheye. At 75° as a true rectilinear lens it seems to have a 3.2mm focal length. But it does have slight distortion, making the image look a little bit like a 3mm fisheye. The vertical FOV is 45° (again 1.6 ratio).

I think this is a waste of time and effort, chasing after FOV and focal length, when considering every lens will have a different projection formula. Focal length will even change if the lens is focused to a different distance, cheap and simple lenses like in webcams certainly do so. The formulas usually measure focal length at infinity, but we sit much closer (or the lens is manufactured with fixed focus to some close distance), so the focal length is reduced, and then the formula used in the tracker becomes wrong.
Maybe the user could choose from several options, each using a different projection formula?
There's some detail here:
http://www.bobatkins.com/photography/technical/field_of_view.html
http://wiki.panotools.org/Fisheye_Projection
After all the industry usually doesn't make lenses with weird and random projections. They tend to come very close to the formula. So this would remove the need for manual calibration.
At worst, maybe change the FOV input to some slider, so users will simply have to reach a position that eliminates incorrect output. They can't always be bothered to know or calculate the camera's FOV to begin with, and will always be confused when entering FOV doesn't produce the expected output.

[sthalik]

considering every lens will have a different projection formula

Camera distortion is parameterized by five coefficients. The issue here is calibrating the camera so that these coefficients are correct.

We can make a directory for distortion coefficient tables, but so far my every attempt resulted in non-working tables.

[sthalik]

@HMRMike please recheck rc14.

With PS3 Eye we could specify calibration file by appending FOV to filename opened, hmm.

[HMRMike]

Awesome!
With the lens at 56 degrees, there is virtually no false yaw response (because the respective.yml was made using this FOV, I guess?). At worst only +-1 in "game data", mostly due to imperfect movement. Even at 75 degrees there is some yaw, but still great, only +-5 (unnoticeable in a game).
All other axes seem accurate as well.

[sthalik]

We're not using calibration for PS3 Eye yet. I'll leave this issue open until we do.

[sthalik]

I have a test build: https://www.dropbox.com/s/38v0955uni7dlg1/opentrack-2.3-rc14-48-g9cc86cf.zip

Please check calibration on 75 fov the following way:

• set FOV value close to 75, like 74 or 76
• press "OK" in dialog, calibration will be disabled
• try how much X affects yaw
• set to 75 the same way
• try X/yaw again

It depends on the quality of the calibration file. It was damn hard to get it to detect the chessboard to begin with.

[sthalik]

Pinging @HMRMike

[HMRMike]

Something isn't working. 3 points are extracted and OK, all other settings checked. But the automatic model position calibration keeps failing, always sets to -65535 mm.
I also don't see the red cross that covers the middle LED when not calibrated. Manually entering values didn't make any difference. In this state, there's no tracking at all.

[sthalik]

There's something wrong with it, yeah. There's no translation Z output which means pose estimation won't converge. It barely works that way. Also focal lengths in calibration files differ by too little. They should be 600 and 900 according to formula, but both are 700-ish.

I've removed calibration for now. Unless it can be done properly there's no point.

[sthalik]

We've achieved all that can be done without accounting for distortion. There are further unrelated changes. If something bad happens in next builds we can reopen the ticket.

Thanks for your feedback.

[HMRMike]

It's all good anyway, even at 75 degrees there's only insignificant distortion induced yaw. And the FOV confusion is over.
Thanks a lot for that!