Added support for spatial navigation (Navigation using arrow keys) and also support for tv remote keys #1885

gibahjoe · 2023-02-15T21:28:07Z

This PR adds support for Spatial navigation which is especially useful when using jellyfin-vue on a TV.

Also support for TV remote keys during playback (Tested with LGTV remote) was also added.

The main focus of this PR is to let TV users select items on their screen using the arrow keys of the remote and also to allow the remote media keys (tested with LGTV magic remote, webOS 5 and 6 but it can be easily expanded for other tv types if their keycodes are different).

sonarcloud · 2023-03-12T21:14:01Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

Ritchie1108 · 2023-04-07T09:30:06Z

so, anybody knows? does this project support smart tv remote?
i want to use it to replace jellyfin-web in jellyfin-tizen

ThibaultNocchi · 2023-04-07T09:45:06Z

I haven't yet gone around to review this PR, we had a lot of work with the recent Vue 3 merge.

I don't mind adding support for keys and remotes, what I don't guarantee is that the client will work on smart TVs. We use a pretty recent software stack, with which TVs usually aren't compatible.

But adding support for keys isn't something I'll block, I just need to take some time :)

Also, @gibahjoe , can you resolve the conflicts with the codebase please? :)

Ritchie1108 · 2023-04-07T09:52:31Z

sorry, im not good at web code, so im not not sure what the browser version this project depends on
maybe chromium m85?

ThibaultNocchi · 2023-04-07T09:55:27Z

It seems at least to need Chromium 87, as Vite specifies, and this isn't checking the other modules we use.

Ritchie1108 · 2023-04-07T10:00:41Z

oops, so sad

gibahjoe · 2023-04-08T07:27:25Z

@ThibaultNocchi The previous vue app worked on TVs (tested on LGTV) but Vite didn't as at 3 months ago when I tested it. I will take a look at it again and resolve these merge conflicts

ferferga · 2023-04-08T10:46:39Z

@gibahjoe Still it's useful if this can be navigated through keyboard for accessibility reasons or TVs that have latest Chromium versions.

There's no harm in supporting as much use cases as possible, as long as the support is good!

gibahjoe · 2023-04-11T14:45:26Z

Update:

I already began work on this a few days back. Unfortunately, the plug-in I used previously doesn't support Vue 3 so I have had to copy over the code and make a couple of adjustments.

Anyway, I should be done by this weekend.

ferferga · 2023-04-15T20:51:22Z

@gibahjoe The other day I was thinking about the media type redesign and it came across to my mind again what to do for this PR.

In general, I think we should have three views for libraries: card grid (the only one right now), coverflow one (the current one for fullscreen music player, which is already useful for touch-based units like Echo Show or Android Auto but it also might be a presentation people like for books and TVs) and a list view (specially for music)

For the home sections, music and that hypothetical coverflow view, we already have full keyboard support through swiper.

Why I'm saying all of this? Maybe before you do further work here, perhaps we should gather feedback from people who might benefit from this the most. I'm not a TV user myself, so all I can do is give my review in the impl details, etc, but maybe not much insight in how it should be UX-wise

… keys on keyboard and remote control. Also add support for play/pause button on webos # Conflicts: # frontend/src/store/playbackManager.ts

sonarcloud · 2023-04-16T14:15:41Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

frontend/src/plugins/spatialNav/spatialNavigation.js

+      currentFocusedElement.blur();
+    }
+
+    focusNscroll(elem);


frontend/src/plugins/spatialNav/spatialNavigation.js

+    return false;
+  }
+
+  focusNscroll(elem);


frontend/src/plugins/spatialNav/index.ts

+    // This might also be split into different directives to handle remove eventlisteners
+    Vue.directive('focus-events', {
+      mounted(el, binding) {
+        for (const [i, key] of Object.keys(binding.value).entries()) {


frontend/src/plugins/spatialNav/spatialNavigation.js

+  for (const candidate of candidates) {
+    var rect = getRect(candidate);
+
+    if (rect) {


jellyfin-bot · 2023-04-16T14:28:56Z

Cloudflare Pages deployment

Latest commit	`ba6e882`
Status	✅ Deployed!
Preview URL	https://jf-vue.pages.dev (https://0d8cb1de.jf-vue.pages.dev)
Type	⚙️ Production

View build logs
View bot logs

gibahjoe · 2023-04-16T16:26:13Z

@ferferga That is fine.

endrl · 2023-04-17T14:20:01Z

Playground for WebOS and Tizen is now open ;)
#1967

endrl · 2023-05-03T16:27:07Z

The question is should be "keyboard navigation" with auto focus and highlight a default feature which is always enabled? Or a build flag? Webos, tizen and any other client(*older) that might want to embed jellyfin-vue needs compatibility hacks anyway

Media Keys
Implementing some media keys for the player(s) might be a good thingKeyboardEvent and MultimediaKeys

// code is empty because of windows...?! good to know

KeyboardEvent: key='MediaPlayPause' | code=''
KeyboardEvent: key='MediaTrackNext' | code=''
KeyboardEvent: key='MediaTrackPrevious' | code=''
KeyboardEvent: key='AudioVolumeMute'  | code=''
KeyboardEvent: key='AudioVolumeDown' | code=''
KeyboardEvent: key='AudioVolumeUp' | code=''

I had a look regarding key code mapping.

I don't know what jellyfin-web has implemented at this point
jellyfin-tizen i found no hacks. See Tizen docs and jellyfin-tizen
jellyfin-webos has basic key code mapping. However the media keys and partly the navigation seems to work out of the box

webos and tizen might be able to inject something like this if really required (Not sure if this really works).

// https://stackoverflow.com/questions/8776543/how-to-catch-event-keycode-and-change-it-to-another-keycode
Object.defineProperty(KeyboardEvent.prototype, 'key', {
    get: function() {
        switch (this.keyCode) {
            case 19: return 'MediaPlayPause';
            case 415: return 'MediaPlayPause';
            default: return this.key
        }
    }
})

ferferga · 2023-05-04T16:27:28Z

Just a heads up for everybody reading: @ThibaultNocchi and me have been discussing this internally.
This might be a good fit to be in core, since key navigation is also important for accesibility concerns. However, in case it gets too "out of scope", I opened a discussion explaining how we want to deal with features that could get too platform/use-case specific.

But again, it might be interesting to have a proof of this wotking in core first (hence all merge conflicts fixed) because it's relevant for desktop users as well.

Ritchie1108 · 2023-05-05T09:34:06Z

The question is should be "keyboard navigation" with auto focus and highlight a default feature which is always enabled? Or a build flag? Webos, tizen and any other client(*older) that might want to embed jellyfin-vue needs compatibility hacks anyway

Media Keys Implementing some media keys for the player(s) might be a good thingKeyboardEvent and MultimediaKeys
// code is empty because of windows...?! good to know

KeyboardEvent: key='MediaPlayPause' | code=''
KeyboardEvent: key='MediaTrackNext' | code=''
KeyboardEvent: key='MediaTrackPrevious' | code=''
KeyboardEvent: key='AudioVolumeMute'  | code=''
KeyboardEvent: key='AudioVolumeDown' | code=''
KeyboardEvent: key='AudioVolumeUp' | code=''
I had a look regarding key code mapping.

I don't know what jellyfin-web has implemented at this point

jellyfin-tizen i found no hacks. See Tizen docs and jellyfin-tizen

jellyfin-webos has basic key code mapping. However the media keys and partly the navigation seems to work out of the box

webos and tizen might be able to inject something like this if really required (Not sure if this really works).
// https://stackoverflow.com/questions/8776543/how-to-catch-event-keycode-and-change-it-to-another-keycode
Object.defineProperty(KeyboardEvent.prototype, 'key', {
    get: function() {
        switch (this.keyCode) {
            case 19: return 'MediaPlayPause';
            case 415: return 'MediaPlayPause';
            default: return this.key
        }
    }
})

https://github.com/jellyfin/jellyfin-web/blob/master/src/scripts/keyboardNavigation.js

jellyfin-bot added this to In progress in Ongoing development Feb 15, 2023

jellyfin-bot added plugins Pull requests that edit or add Nuxt plugins vue Pull requests that edit or add Vue files labels Feb 15, 2023

gibahjoe force-pushed the master branch from 2607054 to b8c3921 Compare February 18, 2023 23:10

ferferga added the discussion needed This PR or issue needs discussion before going further label Mar 14, 2023

jellyfin-bot added the merge conflict Something has merge conflicts label Apr 3, 2023

gibahjoe closed this Apr 16, 2023

gibahjoe force-pushed the master branch from 21d063d to 5ae8928 Compare April 16, 2023 13:58

Ongoing development automation moved this from In progress to Done Apr 16, 2023

jellyfin-bot moved this from Done to In progress in Ongoing development Apr 16, 2023

jellyfin-bot removed plugins Pull requests that edit or add Nuxt plugins vue Pull requests that edit or add Vue files labels Apr 16, 2023

gibahjoe added 2 commits April 16, 2023 15:01

feat (spatial navigation): This allows for navigation using the arrow…

16ee7b2

… keys on keyboard and remote control. Also add support for play/pause button on webos # Conflicts: # frontend/src/store/playbackManager.ts

completed spatial nav

53e20a2

gibahjoe reopened this Apr 16, 2023

jellyfin-bot added vue Pull requests that edit or add Vue files and removed merge conflict Something has merge conflicts labels Apr 16, 2023

github-advanced-security bot found potential problems Apr 16, 2023

View reviewed changes

jellyfin-bot added the merge conflict Something has merge conflicts label Apr 28, 2023

ferferga force-pushed the master branch 3 times, most recently from 5e698fd to 26d3ab7 Compare May 1, 2023 22:03

ferferga mentioned this pull request May 4, 2023

feat(player): Add media key control support #2001

Open

ferferga added the blocked Something depends on another issue or Pull Request label May 4, 2023

ferferga force-pushed the master branch from 2a1743e to a22eecd Compare June 1, 2023 12:29

ferferga force-pushed the master branch 2 times, most recently from bc856ac to 7336bfd Compare August 16, 2023 19:17

ferferga force-pushed the master branch from 75811cc to ad19e26 Compare December 16, 2023 15:48

ferferga force-pushed the master branch 2 times, most recently from f598745 to 00c2c75 Compare December 31, 2023 18:39

ferferga force-pushed the master branch from d4dbd9c to ff9f91e Compare January 10, 2024 11:27

ferferga force-pushed the master branch 8 times, most recently from 937e458 to 6c210e6 Compare April 16, 2024 08:57

ferferga force-pushed the master branch from e907a77 to 3264021 Compare May 8, 2024 01:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for spatial navigation (Navigation using arrow keys) and also support for tv remote keys #1885

Added support for spatial navigation (Navigation using arrow keys) and also support for tv remote keys #1885

gibahjoe commented Feb 15, 2023 •

edited

sonarcloud bot commented Mar 12, 2023

Ritchie1108 commented Apr 7, 2023

ThibaultNocchi commented Apr 7, 2023 •

edited

Ritchie1108 commented Apr 7, 2023

ThibaultNocchi commented Apr 7, 2023 •

edited

Ritchie1108 commented Apr 7, 2023

gibahjoe commented Apr 8, 2023

ferferga commented Apr 8, 2023

gibahjoe commented Apr 11, 2023

ferferga commented Apr 15, 2023

sonarcloud bot commented Apr 16, 2023

jellyfin-bot commented Apr 16, 2023

gibahjoe commented Apr 16, 2023

endrl commented Apr 17, 2023

endrl commented May 3, 2023 •

edited

ferferga commented May 4, 2023

Ritchie1108 commented May 5, 2023 •

edited

Added support for spatial navigation (Navigation using arrow keys) and also support for tv remote keys #1885

Are you sure you want to change the base?

Added support for spatial navigation (Navigation using arrow keys) and also support for tv remote keys #1885

Conversation

gibahjoe commented Feb 15, 2023 • edited

sonarcloud bot commented Mar 12, 2023

Ritchie1108 commented Apr 7, 2023

ThibaultNocchi commented Apr 7, 2023 • edited

Ritchie1108 commented Apr 7, 2023

ThibaultNocchi commented Apr 7, 2023 • edited

Ritchie1108 commented Apr 7, 2023

gibahjoe commented Apr 8, 2023

ferferga commented Apr 8, 2023

gibahjoe commented Apr 11, 2023

ferferga commented Apr 15, 2023

sonarcloud bot commented Apr 16, 2023

jellyfin-bot commented Apr 16, 2023

Cloudflare Pages deployment

gibahjoe commented Apr 16, 2023

endrl commented Apr 17, 2023

endrl commented May 3, 2023 • edited

ferferga commented May 4, 2023

Ritchie1108 commented May 5, 2023 • edited

gibahjoe commented Feb 15, 2023 •

edited

ThibaultNocchi commented Apr 7, 2023 •

edited

ThibaultNocchi commented Apr 7, 2023 •

edited

endrl commented May 3, 2023 •

edited

Ritchie1108 commented May 5, 2023 •

edited