Wrong touch input with different display rotations

[gongzhenjeef]

Hello, I'm using a laptop touchscreen with two external monitor. When I rotate one of my external screen, the touch input on laptop screen and another monitor is also rotated. I suppose the rotation should be stick to the main display@0 for physical connection regardless of other monitor status.

[ben9923]

We might want to search for IOBacklightDisplay instead of IODisplay.

That said, invalid user configuration might cause a built-in display to appear as IODisplay (AppleDisplay) instead of IOBacklightDisplay (AppleBacklightDisplay). Should we fallback to IODisplay in this case?

What do you think @blankmac ?

[alexandred]

We might want to search for IOBacklightDisplay instead of IODisplay.

That said, invalid user configuration might cause a built-in display to appear as IODisplay (AppleDisplay) instead of IOBacklightDisplay (AppleBacklightDisplay). Should we fallback to IODisplay in this case?

What do you think @blankmac ?

This issue is not due to the display itself, rather how we rotate the input. When the internal display of the device is rotated, we manually swap the x and y coordinates of the input before sending it into MT2 simulator. Hence all input is rotated regardless of what display the cursor is currently on.

This is really a bug on Apple’s end. We shouldn’t need to manually swap the coordinates since Apple’s input stack should be smart enough to do that itself. Alas it no longer does (it used to in earlier versions of macOS) so this was the only solution I could come up with to give users with tablet screens a reasonable experience.

If there was a way to query the current display the cursor is on then, theoretically, we should be able to fix this issue by only swapping x and y coordinates when the cursor is on a rotated display. Unfortunately I have no idea how to do that at the moment. Perhaps some deep digging into the display classes is in order sometime in the future...

[ben9923]

We might want to search for IOBacklightDisplay instead of IODisplay.
That said, invalid user configuration might cause a built-in display to appear as IODisplay (AppleDisplay) instead of IOBacklightDisplay (AppleBacklightDisplay). Should we fallback to IODisplay in this case?
What do you think @blankmac ?

This issue is not due to the display itself, rather how we rotate the input. When the internal display of the device is rotated, we manually swap the x and y coordinates of the input before sending it into MT2 simulator. Hence all input is rotated regardless of what display the cursor is currently on.

This is really a bug on Apple’s end. We shouldn’t need to manually swap the coordinates since Apple’s input stack should be smart enough to do that itself. Alas it no longer does (it used to in earlier versions of macOS) so this was the only solution I could come up with to give users with tablet screens a reasonable experience.

If there was a way to query the current display the cursor is on then, theoretically, we should be able to fix this issue by only swapping x and y coordinates when the cursor is on a rotated display. Unfortunately I have no idea how to do that at the moment. Perhaps some deep digging into the display classes is in order sometime in the future...

You are right, but as Apple's input stack won't handle it, we should at least try to improve it...
Don't you think rotating by the built-in display is more correct than rotating by the first display we find (We get the framebuffer from the first IODisplay match we have)?