Add high-frequency touch events to iOS GameView #8092

Mindfulplays · 2023-11-15T23:24:56Z

Fine-grained/high-frequency touch events are available in certain platforms upon request. On iOS, coalesced touch events can be obtained from the GetCoalescedTouch via the passed-in event.

This PR proposes a side-channel OnCoalescedTouchEvent that can be used in lieu or in addition to OnTouchEvent without any performance penalty. I have tested with and without the coalesced touch event including Apple Pencil which produces events > 60 events per second.

Also fixed minor typos here and there (going forward, I will create smaller commit PRs to avoid polluting).

MonoGame 3.8 Release

Update master

Fix Missing Android Package

In other `Platform` classes (such as `SDLGamePlatform` and `AndroidGamePlatform`, `Threading.Run()` is called during `Tick()` or `RunLoop()` (inconsistently, it may be called before or after `Game.Tick()` unfortunately). Fixing this for `iOSGamePlatform`. `WinForms` and perhaps other platforms (Switch?) may suffer from the same issue. Note that texture loading from background thread is discouraged however a lot of computationally intensive operations (reading/decoding the image file streams, allocating CPU buffers) could be done in the background thread and hence is a good practice to move such work alone into the bgthread until we request the actual texture from the main thread.

…o develop

…to develop

…evelop

…to develop

…evelop

…to develop

…evelop

…to develop

Mindfulplays · 2023-11-15T23:26:58Z

@harry-cpp @mrhelmut Could you please help review? thank you!

stromkos · 2023-11-16T02:35:12Z

On the PR itself, there are several regression issues that need to be addressed.

Please collapse the commits.

There are thread timing issues that need to be formally addressed since the entire TouchCollection becomes asynchronous.

Rather than passing an event through the framework for this case, set a listener, hook with callback delagate, for the event and call the handler(a static function elsewhere) in Program.cs or Activity.cs.

That was how I handled the delay issue on Android and Windows.

What are the associated costs:

https://developer.apple.com/documentation/uikit/touches_presses_and_gestures/handling_touches_in_your_view/getting_high-fidelity_input_with_coalesced_touches

Important

Coalesced touches are intended for apps that need the extra precision and can handle the associated costs. Processing coalesced touches means gathering additional data and applying it to your content. If you don’t need the extra precision, continue using the set of touch objects that UIKit passes to the methods of your views or gesture recognizers.

Mindfulplays · 2023-11-16T20:05:57Z

On the PR itself, there are several regression issues that need to be addressed.

Please collapse the commits.

There are thread timing issues that need to be formally addressed since the entire TouchCollection becomes asynchronous.

Rather than passing an event through the framework for this case, set a listener, hook with callback delagate, for the event and call the handler(a static function elsewhere) in Program.cs or Activity.cs.

That was how I handled the delay issue on Android and Windows.

What are the associated costs:

https://developer.apple.com/documentation/uikit/touches_presses_and_gestures/handling_touches_in_your_view/getting_high-fidelity_input_with_coalesced_touches

Important
Coalesced touches are intended for apps that need the extra precision and can handle the associated costs. Processing coalesced touches means gathering additional data and applying it to your content. If you don’t need the extra precision, continue using the set of touch objects that UIKit passes to the methods of your views or gesture recognizers.

Could you add what the regressions are to the PR comments? Not seeing it...

The coalesced events (+processing) is optional (existing apps can choose to subscribe by adding OnCoalescedTouchEvent handler). No extra work is done unless the caller asks for it.

I am not aware of any thread timing issues as the evt holds on to the TouchCollection in memory. There is no round-trip back to the OS or any other blocking main-thread activity. I verified by both stepping through the framework code and the provided iOS documentation.

In case I am missing something, please clarify / provide specific examples.

Mindfulplays · 2023-11-29T19:48:35Z

@harry-cpp @mrhelmut could you please take a look? Thanks!

…evelop

…to develop

tomspilman · 2023-12-13T06:58:26Z

MonoGame.Framework/Input/Touch/TouchLocation.cs

@@ -1,7 +1,7 @@
 #region License
 // /*
 // Microsoft Public License (Ms-PL)
-// MonoGame - Copyright � 2009-2010 The MonoGame Team
+// MonoGame - Copyright � 2009-2010 The MonoGame Team


What happened here @Mindfulplays ?

Codepage issues (apparently, VS Code says UTF-8!), just made it (C) :) note that GH diff shows/has codepage issues in the 'before' too for this file and the other ones, so fixed both.

MonoGame.Framework/Input/Touch/TouchLocation.cs

MonoGame.Framework/Input/Touch/TouchPanel.cs

MonoGame.Framework/Input/Touch/TouchPanelState.cs

tomspilman · 2023-12-13T07:28:30Z

Hey @Mindfulplays

There are some changes to readme.md files in this PR. Is that in other PR somewhere? Are we waiting for that to be merged?

…into develop

Mindfulplays · 2023-12-14T05:46:36Z

Hey @Mindfulplays

There are some changes to readme.md files in this PR. Is that in other PR somewhere? Are we waiting for that to be merged?

Apologies, non-familiarity with gh workflows, accidental, removed.

Mindfulplays · 2023-12-14T05:48:01Z

Please take a look, I resolved your comments.

tomspilman · 2023-12-18T14:51:08Z

Thanks @Mindfulplays

SimonDarksideJ · 2023-12-18T18:01:38Z

I take it @Mindfulplays there is not an Android equivalent path for Android? (as this PR only focuses on iOS)

Mindfulplays · 2023-12-18T18:04:58Z

There is: we can get History of touch events in Android. I will send a separate patch for that.

Similarly we have DirectManipulation in Windows but that might be a moot point as I am not sure if it's useful in the context of Windows tablets as a market compared to mobile. (Thoughts?)

Mindfulplays · 2023-12-18T18:06:49Z

Also am not sure if there is such a thing on Consoles (may be Switch or Steam Deck?), but I don't have access to Consoles SDKs so currently wouldn't be able to dabble in it.

SimonDarksideJ · 2023-12-19T14:53:13Z

If it is available on Android, then it would make sense @Mindfulplays to have parity there.
As for Windows touch, tbh, I'm not sure we need to bother unless it is easy to implement as it is not a touch-first platform like iOS / Android.

As for consoles, only Switch really would be compatible (possibly steam deck) but again, they are not touch-first devices, so I do not think we need to worry about those.

tomspilman · 2023-12-19T19:11:18Z

This is one of those features where it may only be available on a couple of platforms.

We should come up with a way to document what platforms these features work on.

Is there some sort of reference document tag we can use for this?

If not maybe some sort of common language in reference docs like:

This feature is available on iOS, Android.

We would just need to be sure to keep this up to date.

Mindfulplays · 2023-12-20T03:57:34Z

Added Android support + doc in #8109

@harry-cpp

* Fix iOS GamePlatform to run pending background tasks MonoGame#7520 In other `Platform` classes (such as `SDLGamePlatform` and `AndroidGamePlatform`, `Threading.Run()` is called during `Tick()` or `RunLoop()` (inconsistently, it may be called before or after `Game.Tick()` unfortunately). Fixing this for `iOSGamePlatform`. `WinForms` and perhaps other platforms (Switch?) may suffer from the same issue. Note that texture loading from background thread is discouraged however a lot of computationally intensive operations (reading/decoding the image file streams, allocating CPU buffers) could be done in the background thread and hence is a good practice to move such work alone into the bgthread until we request the actual texture from the main thread. * Update docs for sync/test code * Fix typo in comment * Add coalesced touch events to iOS GameView * Add coalesced touch API to TouchPanel* * Fix UTF-8 copyright mistake * Ensure we process coalesced touches only if the client asked for it * Avoid using C#5.0+ features (expression-bodied properties) * Suggested changes * Remove Coalesced API (new) and update existing touch event with a flag for `isHighFrequency`. - Also consistently use `high-frequency` instead of `coalesced` when referring to this new behavior. * Remove `out int` C# 7.0 / use C# 5.0 feature instead * Update based on @harry-cpp comments * Rename to `origin` instead of `upstream` * Remove git stuff * Address PR comments * Change (c) UTF-8 due to codepage issues, my bad. --------- Co-authored-by: Tom Spilman <tom@sickheadgames.com> Co-authored-by: Thao D <thao.mindfulplay@gmail.com> Co-authored-by: Simon (Darkside) Jackson <darkside@zenithmoon.com> Co-authored-by: mindfulplay.build <mindfulplay@appu.app>

@harry-cpp

* Fix iOS GamePlatform to run pending background tasks MonoGame#7520 In other `Platform` classes (such as `SDLGamePlatform` and `AndroidGamePlatform`, `Threading.Run()` is called during `Tick()` or `RunLoop()` (inconsistently, it may be called before or after `Game.Tick()` unfortunately). Fixing this for `iOSGamePlatform`. `WinForms` and perhaps other platforms (Switch?) may suffer from the same issue. Note that texture loading from background thread is discouraged however a lot of computationally intensive operations (reading/decoding the image file streams, allocating CPU buffers) could be done in the background thread and hence is a good practice to move such work alone into the bgthread until we request the actual texture from the main thread. * Update docs for sync/test code * Fix typo in comment * Add coalesced touch events to iOS GameView * Add coalesced touch API to TouchPanel* * Fix UTF-8 copyright mistake * Ensure we process coalesced touches only if the client asked for it * Avoid using C#5.0+ features (expression-bodied properties) * Suggested changes * Remove Coalesced API (new) and update existing touch event with a flag for `isHighFrequency`. - Also consistently use `high-frequency` instead of `coalesced` when referring to this new behavior. * Remove `out int` C# 7.0 / use C# 5.0 feature instead * Update based on @harry-cpp comments * Rename to `origin` instead of `upstream` * Remove git stuff * Address PR comments * Change (c) UTF-8 due to codepage issues, my bad. --------- Co-authored-by: Tom Spilman <tom@sickheadgames.com> Co-authored-by: Thao D <thao.mindfulplay@gmail.com> Co-authored-by: Simon (Darkside) Jackson <darkside@zenithmoon.com> Co-authored-by: mindfulplay.build <mindfulplay@appu.app>

tomspilman and others added 26 commits August 10, 2020 07:39

Merge pull request MonoGame#7291 from MonoGame/develop

b2a8062

MonoGame 3.8 Release

Merge pull request MonoGame#7293 from MonoGame/develop

15a6079

Update master

Merge pull request MonoGame#7295 from MonoGame/develop

f79c288

Fix Missing Android Package

Merge branch 'MonoGame:develop' into develop

55b6a49

Merge branch 'master' of https://github.com/Mindfulplays/MonoGame int…

8c1d1b1

…o develop

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

78ddfbf

…to develop

Merge branch 'MonoGame:develop' into develop

53aed57

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

542b029

…to develop

Merge branch 'MonoGame:develop' into develop

05857ed

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

efcdc0f

…to develop

Merge branch 'develop' of https://github.com/MonoGame/MonoGame into d…

3ad9138

…evelop

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

d71c01c

…to develop

Merge branch 'develop' of https://github.com/MonoGame/MonoGame into d…

fbc00f5

…evelop

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

9185622

…to develop

Update docs for sync/test code

bd61703

Merge branch 'MonoGame:develop' into develop

a697756

Merge branch 'develop' of https://github.com/MonoGame/MonoGame into d…

e23899f

…evelop

Merge branch 'MonoGame:develop' into develop

512c8ab

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

edb0e32

…to develop

Fix typo in comment

2b6fd4c

Add coalesced touch events to iOS GameView

175a6be

Add coalesced touch API to TouchPanel*

64ffa3c

Fix UTF-8 copyright mistake

8af6588

Ensure we process coalesced touches only if the client asked for it

64d9a70

Avoid using C#5.0+ features (expression-bodied properties)

ac30570

Mindfulplays mentioned this pull request Dec 12, 2023

Bring back tests for iOS, update to dotnet 6+, ensure works on device/simulator #8099

Closed

Merge branch 'pr/1' into ios-touch

c103dfe

Mindfulplays changed the title ~~Add coalesced touch events to iOS GameView~~ Add high-frequency touch events to iOS GameView Dec 12, 2023

Mindfulplays added 4 commits December 12, 2023 10:47

Merge branch 'develop' of https://github.com/MonoGame/MonoGame into d…

46758cf

…evelop

Update based on @harry-cpp comments

e22cf99

Rename to origin instead of upstream

3530925

Merge branch 'develop' of https://github.com/Mindfulplays/MonoGame in…

6dd86c1

…to develop

tomspilman reviewed Dec 13, 2023

View reviewed changes

MonoGame.Framework/Input/Touch/TouchLocation.cs Show resolved Hide resolved

tomspilman reviewed Dec 13, 2023

View reviewed changes

MonoGame.Framework/Input/Touch/TouchPanel.cs Outdated Show resolved Hide resolved

tomspilman reviewed Dec 13, 2023

View reviewed changes

MonoGame.Framework/Input/Touch/TouchPanelState.cs Show resolved Hide resolved

Mindfulplays added 4 commits December 13, 2023 15:08

Remove git stuff

0f2e251

Merge branch 'ios-touch' of https://github.com/Mindfulplays/MonoGame …

1264845

…into develop

Address PR comments

2534395

Change (c) UTF-8 due to codepage issues, my bad.

ba0c6a3

tomspilman approved these changes Dec 18, 2023

View reviewed changes

tomspilman merged commit 5fc1d04 into MonoGame:develop Dec 18, 2023
4 checks passed

Mindfulplays mentioned this pull request Dec 19, 2023

Unit Testing framework #7474

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add high-frequency touch events to iOS GameView #8092

Add high-frequency touch events to iOS GameView #8092

Mindfulplays commented Nov 15, 2023 •

edited

Mindfulplays commented Nov 15, 2023

stromkos commented Nov 16, 2023

Mindfulplays commented Nov 16, 2023

Mindfulplays commented Nov 29, 2023

tomspilman Dec 13, 2023

Mindfulplays Dec 14, 2023

tomspilman commented Dec 13, 2023

Mindfulplays commented Dec 14, 2023

Mindfulplays commented Dec 14, 2023

tomspilman commented Dec 18, 2023

SimonDarksideJ commented Dec 18, 2023

Mindfulplays commented Dec 18, 2023

Mindfulplays commented Dec 18, 2023

SimonDarksideJ commented Dec 19, 2023

tomspilman commented Dec 19, 2023

Mindfulplays commented Dec 20, 2023

Add high-frequency touch events to iOS GameView #8092

Add high-frequency touch events to iOS GameView #8092

Conversation

Mindfulplays commented Nov 15, 2023 • edited

Mindfulplays commented Nov 15, 2023

stromkos commented Nov 16, 2023

Mindfulplays commented Nov 16, 2023

Mindfulplays commented Nov 29, 2023

tomspilman Dec 13, 2023

Choose a reason for hiding this comment

Mindfulplays Dec 14, 2023

Choose a reason for hiding this comment

tomspilman commented Dec 13, 2023

Mindfulplays commented Dec 14, 2023

Mindfulplays commented Dec 14, 2023

tomspilman commented Dec 18, 2023

SimonDarksideJ commented Dec 18, 2023

Mindfulplays commented Dec 18, 2023

Mindfulplays commented Dec 18, 2023

SimonDarksideJ commented Dec 19, 2023

tomspilman commented Dec 19, 2023

Mindfulplays commented Dec 20, 2023

Mindfulplays commented Nov 15, 2023 •

edited