iOS sensors have incorrect orientation

[celyk]

Tested versions

v4.2.1.stable.official [b09f793]

System information

iPadOS 17.4, iPad 9th gen

Issue description

I'm using SCREEN_SENSOR orientation to automatically rotate my app, and sensor vectors (gravity, accelerometer, magnetometer, gyro) don't maintain their orientation relative to the app. This causes them to point in unpredictable directions. Portrait is the only mode that works properly.

The relevant code is here. According to this comment, it was meant to correct their coordinate system by a rotation.

godot/platform/ios/godot_view.mm

Lines 452 to 477 in 9bc49a6

	switch (interfaceOrientation) {
	case UIInterfaceOrientationLandscapeLeft: {
	DisplayServerIOS::get_singleton()->update_gravity(-gravity.y, gravity.x, gravity.z);
	DisplayServerIOS::get_singleton()->update_accelerometer(-(acceleration.y + gravity.y), (acceleration.x + gravity.x), acceleration.z + gravity.z);
	DisplayServerIOS::get_singleton()->update_magnetometer(-magnetic.y, magnetic.x, magnetic.z);
	DisplayServerIOS::get_singleton()->update_gyroscope(-rotation.y, rotation.x, rotation.z);
	} break;
	case UIInterfaceOrientationLandscapeRight: {
	DisplayServerIOS::get_singleton()->update_gravity(gravity.y, -gravity.x, gravity.z);
	DisplayServerIOS::get_singleton()->update_accelerometer((acceleration.y + gravity.y), -(acceleration.x + gravity.x), acceleration.z + gravity.z);
	DisplayServerIOS::get_singleton()->update_magnetometer(magnetic.y, -magnetic.x, magnetic.z);
	DisplayServerIOS::get_singleton()->update_gyroscope(rotation.y, -rotation.x, rotation.z);
	} break;
	case UIInterfaceOrientationPortraitUpsideDown: {
	DisplayServerIOS::get_singleton()->update_gravity(-gravity.x, gravity.y, gravity.z);
	DisplayServerIOS::get_singleton()->update_accelerometer(-(acceleration.x + gravity.x), (acceleration.y + gravity.y), acceleration.z + gravity.z);
	DisplayServerIOS::get_singleton()->update_magnetometer(-magnetic.x, magnetic.y, magnetic.z);
	DisplayServerIOS::get_singleton()->update_gyroscope(-rotation.x, rotation.y, rotation.z);
	} break;
	default: { // assume portrait
	DisplayServerIOS::get_singleton()->update_gravity(gravity.x, gravity.y, gravity.z);
	DisplayServerIOS::get_singleton()->update_accelerometer(acceleration.x + gravity.x, acceleration.y + gravity.y, acceleration.z + gravity.z);
	DisplayServerIOS::get_singleton()->update_magnetometer(magnetic.x, magnetic.y, magnetic.z);
	DisplayServerIOS::get_singleton()->update_gyroscope(rotation.x, rotation.y, rotation.z);
	} break;
	}

The 3rd case is wrong and should have been (-x,-y,z) in order to rotate 180 degrees in the xy plane. The other 2 cases may have worked at the time, but not anymore.

Steps to reproduce

Export the MRP for iOS. The 3D arrow should point down no matter the orientation.

You can also use the official sensors demo: https://github.com/godotengine/godot-demo-projects/tree/master/mobile/sensors

Minimal reproduction project (MRP)

down-MRP.zip

[akien-mga]

CC @bruvzg

[celyk]

Some footage of the problem. First, I show that on Android the gravity arrow always points down in real life. At 16 seconds I show what happens on iOS.

sensor_bug_iOS.mp4

[bruvzg]

The relevant code is here. According to #8101 (comment), it was meant to correct their coordinate system by a rotation.

I think iPhones and iPads might have different base orientations (or maybe it's different on different models), at least on 10 gen. iPad orientation seems to be flipped (not only sensors but orientation in general).

[celyk]

I'm not able to find any sources on that. What appears to be happening is that the sensors are being rotated the wrong way, which is why the combined rotation makes the gravity vector point straight up.

Tbh I don't think a different base orientation can explain it. Wouldn't that result in all the sensors being rotated 90 degrees in Godot? If your iPad produces different results to mine please say. I also tested on an iPhone 13 Pro and it's the same.

Here's how Apple defines the device coordinate system from https://developer.apple.com/documentation/coremotion/cmmotionmanager

[bruvzg]

I think iPhones and iPads might have different base orientations

Or maybe it's just UIDeviceOrientation and UIInterfaceOrientation mixed up in some places. I'll test in on different simulators and device.

Important

Notice that UIDeviceOrientationLandscapeRight is assigned to UIInterfaceOrientationLandscapeLeft and UIDeviceOrientationLandscapeLeft is assigned to UIInterfaceOrientationLandscapeRight. The reason for this is that rotating the device requires rotating the content in the opposite direction.

[bruvzg]

Seems like there are two issues actually, sensors are incorrect if orientation is controlled by sensor (probably on all devices), and default landscape orientation (if set to fixed) is upside-down (this might be specific for different devices).