Swipe gestures support PoC #2355
Conversation
| //TODO: App Expose activates if swipe down with 3-fingers right after AltTab activation. There is no issue if wait for some time. | ||
| //TODO: enable/disable trackpad usage in settings | ||
| class TrackpadEvents { | ||
| //TODO: Should we add a sensetivity setting instead of these magic numbers? |
There was a problem hiding this comment.
We can either hardcode such numbers if they are stable in macOS over time, or we may be able to read the system settings, like we read the key repeat rate today, or other System Settings settings.
| @@ -0,0 +1,113 @@ | |||
| import M5MultitouchSupport | |||
|
|
|||
| //TODO: patch M5MultitouchSupport pod to make it not to crash after sleep. See https://github.com/mhuusko5/M5MultitouchSupport/issues/1 | |||
There was a problem hiding this comment.
I'm a bit worried to rely on a third-party library that hasn't been updated in 8 years, and that has a crash on wake that has only been found this year.
Is there no way to roll our own code for multi-fingers gesture? Is it too rough when using the AppKit APIs?
There was a problem hiding this comment.
Update: i just searched online for this, and there is no information out there. I understand better why you went with this library. Classic Apple move to not properly support such a classic interaction decades after it's been added to macOS.
I've updated the issue on the M5MultitouchSupport repo. Maybe we could add something like this to fix the issue.
There was a problem hiding this comment.
I don't want to add a 3-rd party library either but it's just a PoC.
We could patch M5MultitouchSupport as I did it for Touch-Tab https://github.com/ris58h/Touch-Tab/blob/master/patches/M5MultitouchSupport%2B1.1.0.diff (it works fine) or use any other library.
Seems like https://github.com/Kyome22/OpenMultitouchSupport is a good candidate.
P.S.:
Code in both libraries is nearly identical (M5MultitouchSupport vs OpenMultitouchSupport) but OpenMultitouchSupport checks if MTDeviceIsRunning before releasing the device.
There was a problem hiding this comment.
It turned out that OpenMultitouchSupport has the same issue Kyome22/OpenMultitouchSupport#2 It seems like we have to patch anyway.
Should I commit changes in Pods dir after a new pod is installed?
| } else { | ||
| if isHorizontal { | ||
| DispatchQueue.main.async { App.app.showUi() } | ||
| activated = true |
There was a problem hiding this comment.
Do we need to track this bool actually? Maybe App.app.appIsBeingUsed is enough?
Shouldn't the user be able to open AltTab with alt+tab, but then 3-fingers swipe left or right?
There was a problem hiding this comment.
We need to track that the UI was triggered by swiping because end of the swipe (number of fingers not equal to 3 in our case) hides the UI. If a user opens the UI from menu or by shortcut, every mouse move except 3-finger swipe will close the UI.
There was a problem hiding this comment.
Maybe the following logic could work?
We track that we reached 3 fingers either:
- Before AltTab was opened with the trackpad gesture
- After it was opened with the keyboard shortcut
From that point on, any < 3 fingers gesture means we should focus the window and close UI.
There was a problem hiding this comment.
We could track what caused the UI to be shown by adding a parameter to showUI method and storing it for future checks.
|
Here are comments on your message in the original ticket.
Today when the user sets their keyboard shortcut to
I suggested how to implement the preference here.
I added a comment there. Maybe we could dig into that workaround?
Maybe it's ok with 3 fingers for now. Maybe no-one needs 4 fingers. And 2 fingers is definitely not a good idea for a global trigger.
Again, we could maybe disable that with private API calls. Alternatively, we could maybe absorb some of the gesture events like we absorb keyboard events (so that the apps under AltTab don't get a Furthermore, I want to add that in this PR, you have implemented left/right swipes to move around the thumbnails. I think ideally we would want to implement 360 movement to move around the grid of thumbnails if the user has many. |
I couldn't find the way to do it using private API used in M5 library because it's just a callback. Maybe we could use
Just to make it clear: do you want to add
I've answered there.
Yeah, 2-fingers is not an option here.
It's strange to me that App Expose doesn't activate if I wait for a couple of seconds. I've answered in the code comments about disabling native behavior.
What is 360 movement? I've already implemented thumbnails navigation in all directions (left/right/up/down). |
Yes. That way people can decide which windows they will see in that mode. Originally I was thinking people could bind different swipes (e.g. 3 fingers vs 4 fingers) to different windows to show. A bit like how
The way I would describe it is: imagine that moving your 3-fingers gesture on the trackpad moves a hidden mouse cursor. When you move in 2D space, in 360 degrees of freedom, you're now moving around precisely. Now we make that hidden mouse cursor select thumbnails. It's very different from only being able to go in 1 direction at a time. You can move in diagonal. To imagine the difference, imagine that you are inside a folder, and in grid view. Now imagine wanting to go to up-and-left to a particular file. Either you swipe directly towards the file (i.e. 360 movement), and the selection roughly follows your motion, or you would have to swipe horizontally, then vertically, a distance that potentially quite long, and see your selection move in a L pattern. If you use a Windows machine, and check out the built-in alt+tab UX with a trackpad, you can experience how great the 360 experience is. On that note, currently we can scroll up and down with the mouse wheel or 2-fingers trackpad, when the grid has a lot to display. That code should probably be updated to accomodate the new 3-fingers interactions. |
Oh, I got it.
I've tried to allow both scrolling and moving with trackpad with 2 and 3 finger swipes accordingly but it was clunky. Unfortunately I don't have a Windows laptop right now to check. I believe we should mimic Windows behavior if it's possible. |
Windows behavior is actually pretty bad in that regard. I generally find their UX very good, but for some reason their scrolling is horrible. It's row-by-row instead of pixel-by-pixel, so it can be confusing to the user. They also don't do any form of "look ahead" like they would do in a platformer video game where we want to see a bit of what's ahead, not be at the bleeding edge. Finally, if you own Parallels, or maybe even with the free VirtualBox or BootCamp, you can install a free Windows 10 VM these days, from Microsoft website. It's an easy way to play around with Windows on a mac, for free. |
|
@lwouis it almost works now. Things to do:
I believe we should go with MVP. 360 movement and |
| } | ||
|
|
||
| //TODO: controls in Gesture tab aren't centered like in Shortcut tab | ||
| //TODO: 'On release do nothing' doesn't work. See ShortcutStylePreference.focusOnRelease usage and ATShortcut.shouldTrigger usage |
There was a problem hiding this comment.
For keyboard shortcut 'On release do nothing' is implemented via skipping shortcut action. I haven't found workaround to the swipe yet. We could just not support such option for swiping.
This app has no deadline pressure. We can take time to polish things before shipping 👍 |
|
Hi just wondering, if there is an update on this feature yet? :) |
|
@Shahriarnz14 I haven't touched it for a while - I use Touch-Tab instead. The current state of the PR is described in this comment. I'm new to macOS development so there are some issues:
|
|
@ris58h I just checked out the branch, and played with it. You've done some really good work. The feature works well. I'm really impressed you were able to implement the event tracking without the third-party library. I remember trying in the past and hitting a wall, so I'm really impressed. Thank you for all the work here! 🙇♂️ It's very promising, but there needs to be a few more things before we can ship it. In addition to the checklist you shared above, I would add one: clarifying conflicts/issues with the native gestures to the user. For instance, should we have AltTab use gestures by default? Would it work on a default macOS install? Would it conflict with other gestures? We probably should add some form of tooltip/warning when the user could get a broken experience. We could (probably) read the state of some of the System Settings, and depending on the situation, warn the user to toggle some things off to not conflict with AltTab. Some of the TODOS you've written in the PR seem to show non-obvious conflicts, for instance:
|
|
I'm also interested in this PR. Any chance there's an update on this? |
|
@billjcnickolas it was just a PoC. I'm stuck on some issues but I'm not going to spent my time on them anymore. |
|
@ris58h Thank you for your response. It is indeed a really good PoC and I feel like it's really close to completion. I'm not familiar with swift development at all otherwise would've wanted to help. |
lwouis
force-pushed
the
master
branch
2 times, most recently
from
796a3c3
to
a67d123
Compare
| fileprivate let accVelXThreshold: Float = 0.05 | ||
| fileprivate let accVelYThreshold: Float = 0.075 | ||
| fileprivate var accVelX: Float = 0 | ||
| fileprivate var accVelY: Float = 0 |
There was a problem hiding this comment.
| fileprivate let accVelXThreshold: Float = 0.05 | |
| fileprivate let accVelYThreshold: Float = 0.075 | |
| fileprivate var accVelX: Float = 0 | |
| fileprivate var accVelY: Float = 0 | |
| fileprivate let accVelXThreshold: Float = 0.03 | |
| fileprivate let accVelYThreshold: Float = 0.06 | |
| fileprivate var accVelX: Float = 0.35 | |
| fileprivate var accVelY: Float = 0.125 |
Thanks @lwouis for your work! I also tried your touch-pad and found the same problem with this PR. The sensitivity is not matched with Windows (in my experience). I tried different numbers and found that 0.03, 0.06, 0.35, and 0.125 are more sensitive and like how Windows works.
Context