Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tizen: Playback won't start after application is hidden during playback #1184

Open
peijkelhardt opened this issue Nov 24, 2022 · 10 comments
Open

Comments

@peijkelhardt
Copy link

peijkelhardt commented Nov 24, 2022

We have an issue on certain Tizen devices (verified on 2018, 2019 & 2020 models). 2017, 2021 & 2022 models were also tested, but the behaviour differs. The latter don't have issues when the application is restored.

Tizen devices support this Multitasking feature, which allows to store the application state when switching to another application. Whenever the application is reopened, this state should be restored. What we've implemented, is that we stop the player when the application is hidden (using player.stop()) and whenever the application is restored, we try to resume playback using player.loadVideo().

I've collected some logs when setting RxPlayer.LogLevel = "DEBUG" and gathered some relevant data:

Start playback using loadVideo

player.loadVideo({
    transport: "dash",
    url: "https://{redacted}GlobalManifest.mpd",
    keySystems: [{
        type: "widevine",
        ...
    }],
    autoPlay: true,
    enableFastSwitching: false,
    onCodecSwitch: "reload",
    textTrackMode: "html",
    textTrackElement: document.createElement("div"),
});

Logged data with API prefix:

Calling loadvideo https://{redacted}GlobalManifest.mpd dash
API: playerStateChange event LOADING
API: DRM session cleaned-up with success!
API: current playback timeline: timeupdate
API: current playback timeline: loadedmetadata
API: current playback timeline: seeking
API: current playback timeline: play
API: current playback timeline: ratechange
API: current playback timeline: seeked
API: current playback timeline: canplay
API: playerStateChange event LOADED
API: playerStateChange event PLAYING
API: current playback timeline: ratechange
API: current playback timeline: timeupdate

Then, we switch to another application, which fires this visibilitychange event. In the handler, we stop playback using player.stop():

player.stop();

After this call, 2 more logs will be visible:

API: playerStateChange event STOPPED
API: DRM session cleaned-up with success!

After a short period (10 - 20 seconds), we switch back to our app & again, the visibilitychange event fires, but this time we try to resume playback using player.loadVideo():

player.loadVideo({
    transport: "dash",
    url: "https://{redacted}GlobalManifest.mpd",
    keySystems: [{
        type: "widevine",
        ...
    }],
    autoPlay: true,
    enableFastSwitching: false,
    onCodecSwitch: "reload",
    textTrackMode: "html",
    textTrackElement: document.createElement("div"),
});

Again the logs with API prefix have been collected:

Calling loadvideo https://{redacted}GlobalManifest.mpd dash
API: playerStateChange event LOADING
API: DRM session cleaned-up with success!
API: current playback timeline: timeupdate
API: current playback timeline: loadedmetadata
API: current playback timeline: seeking
API: current playback timeline: ratechange
API: current playback timeline: timeupdate
API: current playback timeline: seeked
API: current playback timeline: canplay
API: current playback timeline: ratechange
API: current playback timeline: timeupdate

The player doesn't seem to get into next state, as it stays in LOADING state, and keeps firing timeupdate events

@peaBerberian
Copy link
Collaborator

Hi and sorry for the late response.

The necessary conditions for a LOADED state on Tizen are (

export default function emitLoadedEvent(
):

  • not rebuffering (data is present around the current position)
  • not freezing (playback does not seem to stay in place for too long)
  • readyState >= 3 (which we seem to have here because the "canplay" event has been received)
  • currenRange !== null (which means that we have data around the current position - that we more or less already verified by checking rebuffering)

So the main reason for staying in a LOADING state to me would be the browser telling the player that no data is loaded yet around the current position (which would be weird as we received a canplay event, hinting to the contrary).

Could you copy paste the full current playback timeline log after that canplay event? It should tell us many properties such as if we're considered in rebuffering and it should even draw the content of the current buffer on the following line.

@peijkelhardt
Copy link
Author

These are the logs collected from the initial session:

API: Calling loadvideo https://{redacted}GlobalManifest.mpd dash
API: current media element state tick event init position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 0
API: playerStateChange event LOADING
API: DRM session cleaned-up with success!
API: current media element state tick event timeupdate position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 0
API: current playback timeline:

^0 
timeupdate
API: current media element state tick event loadedmetadata position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 1
API: current playback timeline:
1669709057.38|==0.67==|1669709058.06
                  ^0 
loadedmetadata
API: current media element state tick event internal-seeking position 1669709058.906185 seeking true internalSeek 1669709058.906185 rebuffering true freezing false ended false paused false playbackRate 1 readyState 1
API: current playback timeline:
1669709057.38|==1.63==|1669709059.02
                  ^1669709058.906185 
seeking
API: current media element state tick event play position 1669709058.906185 seeking true internalSeek 1669709058.906185 rebuffering true freezing false ended false paused false playbackRate 0 readyState 1
API: current playback timeline:
1669709057.38|==1.63==|1669709059.02
                  ^1669709058.906185 
play
API: current media element state tick event ratechange position 1669709058.906185 seeking true internalSeek 1669709058.906185 rebuffering true freezing false ended false paused false playbackRate 0 readyState 1
API: current playback timeline:
1669709057.38|==1.91==|1669709059.30
                  ^1669709058.906185 
ratechange
API: current media element state tick event timeupdate position 1669709057.455 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 0 readyState 3
API: current playback timeline:
1669709057.38|==1.91==|1669709059.30
                  ^1669709057.455 
timeupdate
API: current media element state tick event seeked position 1669709057.455 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 3
API: current playback timeline:
1669709057.38|==1.91==|1669709059.30
                  ^1669709057.455 
seeked
API: current media element state tick event canplay position 1669709057.455 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 3
API: current playback timeline:
1669709057.38|==1.91==|1669709059.30
                  ^1669709057.455 
canplay
API: current media element state tick event ratechange position 1669709057.455 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 3
API: current playback timeline:
1669709057.38|==1.91==|1669709059.30
                  ^1669709057.455 
ratechange
API: playerStateChange event LOADED
API: playerStateChange event PLAYING
API: current media element state tick event timeupdate position 1669709058.6 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 3
API: current playback timeline:
1669709055.46|==8.79==|1669709064.26
                  ^1669709058.6 
timeupdate
API: current media element state tick event timeupdate position 1669709059.51 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==11.51==|1669709066.97
                   ^1669709059.51 
timeupdate
API: current media element state tick event timeupdate position 1669709060.483 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==15.35==|1669709070.82
                   ^1669709060.483 
timeupdate
API: current media element state tick event timeupdate position 1669709061.482 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==19.19==|1669709074.65
                   ^1669709061.482 
timeupdate
API: current media element state tick event timeupdate position 1669709062.483 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==21.11==|1669709076.58
                   ^1669709062.483 
timeupdate
API: current media element state tick event timeupdate position 1669709063.469 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==21.11==|1669709076.58
                   ^1669709063.469 
timeupdate
API: current media element state tick event timeupdate position 1669709064.456 seeking false internalSeek null rebuffering false freezing false ended false paused false playbackRate 1 readyState 4
API: current playback timeline:
1669709055.46|==23.03==|1669709078.49
                   ^1669709064.456 
timeupdate
API: playerStateChange event STOPPED
API: DRM session cleaned-up with success!

And these after switching back from another application:

API: Calling loadvideo https://{redacted}GlobalManifest.mpd dash
API: current media element state tick event init position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 0
API: playerStateChange event LOADING
API: DRM session cleaned-up with success!
API: current media element state tick event timeupdate position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 0
API: current playback timeline:

^0 
timeupdate
API: current media element state tick event loadedmetadata position 0 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 1
API: current playback timeline:

^0 
loadedmetadata
API: current media element state tick event internal-seeking position 1669710579.54931 seeking true internalSeek 1669710579.54931 rebuffering true freezing false ended false paused true playbackRate 1 readyState 1
API: current playback timeline:
1669710578.02|==0.15==|1669710578.17
                                    ^1669710579.54931 
seeking
API: current media element state tick event ratechange position 1669710579.54931 seeking true internalSeek 1669710579.54931 rebuffering true freezing false ended false paused true playbackRate 0 readyState 1
API: current playback timeline:
1669710578.02|==0.19==|1669710578.21
                                    ^1669710579.54931 
ratechange
API: current media element state tick event seeked position 1669710578.095 seeking false internalSeek null rebuffering true freezing false ended false paused true playbackRate 0 readyState 3
API: current playback timeline:
1669710578.02|==0.51==|1669710578.54
                  ^1669710578.095 
seeked
API: current media element state tick event canplay position 1669710578.095 seeking false internalSeek null rebuffering true freezing false ended false paused true playbackRate 1 readyState 3
API: current playback timeline:
1669710578.02|==0.51==|1669710578.54
                  ^1669710578.095 
canplay
API: current media element state tick event ratechange position 1669710578.095 seeking false internalSeek null rebuffering true freezing false ended false paused true playbackRate 1 readyState 3
API: current playback timeline:
1669710578.02|==1.15==|1669710579.17
                  ^1669710578.095 
ratechange
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 3
API: current playback timeline:
1669710576.10|==5.75==|1669710581.86
                  ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==12.51==|1669710588.62
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==17.27==|1669710593.38
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==21.11==|1669710597.21
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==21.11==|1669710597.21
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: current media element state tick event timeupdate position 1669710578.095 seeking false internalSeek null rebuffering false freezing false ended false paused true playbackRate 1 readyState 4
API: current playback timeline:
1669710576.10|==23.03==|1669710599.13
                   ^1669710578.095 
timeupdate
API: playerStateChange event STOPPED
API: DRM session cleaned-up with success!

@peaBerberian
Copy link
Collaborator

Ok, so what's weird here is that we're not rebuffering either...

Maybe the media element's duration stays at 0? That's another condition:

if (!shouldValidateMetadata() || mediaElement.duration > 0) {

If it's not that, we may need to put break points or console.logs in that file to see what condition we do not pass.

@peijkelhardt
Copy link
Author

It doesn't seem to enter the emitLoadedEvent function at all. I've added some console.logs in this function, but they never fire in the second scenario.

@peijkelhardt
Copy link
Author

peijkelhardt commented Dec 1, 2022

When setting some more console.logs, it appears that this is the last function which was called:

tryCatch(() => castToObservable(mediaElement.play()), undefined)

Note that Tizen 2019 has a Chromium M63 browser

Edit:
The mediaElement remains in paused state, therefore the play event may never be fired

@peaBerberian
Copy link
Collaborator

Ok thanks!

So it might be that the promise returned by play never resolves here...
Have you set autoPlay to true?

We could try to see if we reach the LOADED state without. Then we could try calling play later, maybe this Tizen issue is based on timing.

@peijkelhardt
Copy link
Author

Yes, we've set autoPlay to true in loadVideo.

When setting this value to false, the LOADED state is indeed reached. Then, when calling player.play(), similar behaviour can be seen. These timeupdate logs are still logging paused state true

@peijkelhardt
Copy link
Author

A small update:

I've modified the RxPlayer code a bit, in order to support setting a new video element. Whenever the Tizen TV is reactivated, I remove the current video element and add a new one to the DOM & RxPlayer:

_proto.setVideoElement = function setVideoElement(videoElement) {
    this.videoElement = videoElement;
}

After this change, the regular code path gets fired (starting from loadVideo) and playback can be resumed!

So I guess, something is wrong with the video element once the application is moved into the background.

@peaBerberian
Copy link
Collaborator

OK interesting. Changing the media element this way may be risky because an RxPlayer considers that it is always linked to the same media element.
Maybe the proper way would be to re-create a new RxPlayer associated to a separate media element.

@peijkelhardt
Copy link
Author

Yes, that might be the best option right now.

I've tested it, by disposing the player first (calling player.dispose()), and creating a new instance of the player (together with a new video element).

After all this, playback starts as expected 😄

So this may be a browser/Tizen issue instead I guess.

Anyway, thanks for your help & feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants