v3.30.0

Release v3.30.0 (2023-03-07)

• 🔍 Overview
• 📑 Changelog
• Forced subtitles
• updateContentUrls API
• For encrypted contents: getKeySystemConfiguration API
• An experimental debug element
• DASH endNumber attribute handling
• API deprecation
• No dependency on RxJS in the RxPlayer source code anymore

🔍 Overview

The v3.30.0 release is, at last, available!

It's a relatively big release which took a long time to test and finish, mainly because it internally completely removes our dependency on the RxJS library. This is the end of a long time effort started with the v3.26.1 (2021-09-14), whose goal is mainly to improve the approachability of the code and the debugging experience.

Note that the v4.0.0-beta.0 pre-release already included an early version of many features introduced here, so you might already know some of them. On that matter, we will release soon after this release a v4.0.0-beta.1 which re-implements future v4 features on top of this release.

The v3.30.0 release brings:

• A new updateContentUrls method allowing to change the Manifest's URL, as long as it represents the same content.

• "forced" narrative subtitles handling.

• DRM: A new getKeySystemConfiguration API which allows to obtain the complete key system configuration obtained from the browser

• DASH: The SegmentTemplate@endNumber attribute is now handled

• A new createDebugElement experimental API (behind the new "DEBUG_ELEMENT" experimental feature) was added, allowing to show debugging information.

• Panasonic 2019 TVs are now properly supported. We had some issues before with content with DRMs

• LG TV: Contents with DRM are now also better supported

• A new BUFFER_FULL_ERROR avoiding mechanism, trying to limit some of such (rare) errors seen in the wild

📑 Changelog

Features

• Add updateContentUrls API, allowing to update the Manifest's URL during playback [#1182]
• DASH: implement forced-subtitles, adding the forced property to the audio tracks API and selecting by default a forced text track linked to the audio track's language if present [#1187]
• DRM: add the getKeySystemConfiguration method to the RxPlayer [#1202]
• add experimental DEBUG_ELEMENT feature and createDebugElement method to render a default debugging HTML element [#1200]

Deprecated

• Deprecate the getVideoLoadedTime method which can be easily replaced (see Deprecated method documentation)
• Deprecate the getVideoPlayedTime method which can be easily replaced (see Deprecated method documentation)
• Deprecate the transportOptions.aggressiveMode option
• DRM: Deprecate the keySystems[].onKeyStatusesChange callback as no good use case was found for it.

Bug fixes

• Fix segment requesting error when playing a DASH content without an url and without BaseURL elements [#1192]
• API: Stop sending events if the content is stopped due to a side-effect of one of the event handler [#1197]
• text-tracks/ttml: fix inconsistent line spacing when resizing the textTrackElement [#1191]
• DRM: Fix race condition leading to a JS error instead of a NO_PLAYABLE_REPRESENTATION [#1201]
• DRM/Compat: Renew MediaKeys at each loadVideo on all WebOS (LG TV) platforms to work around issues [#1188]

Other improvements

• DASH: better detect closed captions [#1187]
• DASH: handle endNumber DASH attribute [#1186]
• DASH: Do not merge AdaptationSet with role "main" anymore [#1214]
• DASH: parse transferCharacteristics property in the MPD to better detect hdr [#1212]
• Support encrypted contents on Panasonic 2019 TVs [#1226]
• Better handle SourceBuffer's QuotaExceededError, responsible for MediaError with the BUFFER_FULL_ERROR code [#1221]
• API: send available...TracksChange events in the very unlikely scenario where tracks are added after a manifest update [#1197]
• Completely remove RxJS dependency from the RxPlayer's source code [#1193]
• DRM: Request PR recommendation when PlayReady is asked and try default recommendation robustnesses [#1189]

Forced subtitles

Note: this feature was first presented in the v4.0.0-beta.0 beta release

Forced subtitles, also referred as forced narrative or just narrative subtitles are subtitles which are meant to be displayed by default when no other subtitle track is selected.

They allow to clarify audio or visual cues that might not be clear for the user, by providing text translation to:

• clarification of audio when it is not clearly audible (for example due to a strong accent or due to distorted audio)
• foreign - or even sometimes invented (e.g. Klingon) - languages spoke in films
• other types of communication which might not be easily understood (a frequent example would be sign language)
• text in foreign languages present on the video track

Screenshot: french forced narrative subtitles translating what the character is saying. The spoken language here is Klingon which few people know how to speak (besides Klingons of course - and yes, I just repeated the same joke on both release notes).

In previous RxPlayer versions, forced subtitles were treated as any other type of text track, with no mean to identify them.
This version adds a new forced property to text tracks described through text tracks API (the getAvailableTextTracks and getTextTrack methods, the availableTextTracksChangeandtextTrackChange` events) to help you identify which text track is a forced one, as you most likely don't want to list it with other regular tracks:

const textTracks = rxPlayer.getAvailableTextTracks();
const textTracksToDisplay = textTracks.filter(t => !t.forced);

Also, the RxPlayer will for now automatically select a forced text track by default (instead of no text track) if one is present for the same language than the default audio track chosen - unless text tracks are explicitely disabled, for example by setting null in the preferredTextTracks.
This means that - if you're using forced narrative subtitles - you might want to be careful if switching the audio track to also switch the text track if a forced one exist in that new language or if the old one is not adapted anymore.

updateContentUrls API

Note: this feature was first presented in the v4.0.0-beta.0 beta release

Some applications recently introduced us to a new need: being able to change the Manifest's URL while the content is playing.
The URL would represent the same content, it's here just a change of URL in itself that is wanted. For example, a usage would be to switch from an overloaded CDN during playback or to change a token present in the URL.

The v3.30.0 gives an answer to these needs by introducing a new method: updateContentUrls.
It allows to update the URL(s) of the Manifest, and even to trigger a manifest refresh directly, through a simple call:

rxPlayer.updateContentUrls([newUrl], {
  // trigger the refresh immediately
  refresh: true,
});

As usual it has been properly documented in its own documentation page.

For encrypted contents: getKeySystemConfiguration API

Note: this feature was first presented in the v4.0.0-beta.0 beta release

Encryption-related issues are the most frequent category of issues currently investigated by the RxPlayer team and its associated applications at least made by Canal+ and our partners.

In that context, we missed a crucial API allowing to facilitate encryption-related debugging on various platforms: One that would allow to indicate easily the current key system configuration relied on.

Particularly the robustness level: PlayReady SL3000 or SL2000? Widevine L1, L2 or L3? And so on.

That's why this release adds the getKeySystemConfiguration API which returns both the actual key system string and the
MediaKeySystemConfiguration currently relied on.

I put there an emphasis on "actual" because, in opposition to the getCurrentKeySystem API, which is now removed, it is the name actually reported by the MediaKeySystemAccess that is here reported, whereas getCurrentKeySystem returned the exact same string than the type originally set on the keySystems option - an arguably less useful value when the RxPlayer could have made translations in between.

An experimental debug element

Applications relying on the RxPlayer often include a special element displaying various playback-related metrics, used for debugging.

That debugging element generally depended on RxPlayer API as well as on attributes of the media element.
The fact th...

v4.0.0-beta.0

Release v4.0.0-beta.0 (2023-01-27)

Quick Links:
📖 API documentation - ⏯ Demo - 🎓 Migration guide from v3

• 🔍 Overview
• A new player state: "FREEZING"
• A more flexible track API
• Improved Representation selection
• Removal of deprecated options and methods
• More expressive decryption options
• videoResolutionLimit constructor option
• Forced subtitles
• DASH endNumber attribute handling
• Automatic reloading when the content changes
• updateContentUrls API
• For encrypted contents: getKeySystemConfiguration API
• No dependency on RxJS in the RxPlayer source code anymore
• 📑 Changelog

🔍 Overview

The first v4.0.0 beta version, v4.0.0-beta.0 is finally here!
The focus of this release was to provide a better API in the now more evolved streaming landscape, especially to better handle DASH multi-Period contents - which are becoming more and more frequent (thanks to new server-side features such as ad-switching, key rotation, live contents with multiple track lists depending on the program etc.). Another main point of this new major release was to clean-up deprecated features from the API, some preventing us from doing advanced work we plan to make in the future.

This is a very big release in terms of changes but don't be scared! This beta version represents a v4 API which though it can be seen as stable (after a less stable, mostly internal, alpha phase), can still evolve before its official release if issues with it are spotted. Also, new v3 releases with bug fixes, improvements and even features will still be made for the foreseeable future.

The goal of this first open beta release is too give you time to make the switch, ask questions, propose changes and report issues that you see with it.
Because we want to make the migration experience as painless as possible, we wrote a complete migration guide, accessible in our documentation pages for the v4.

As the changelog is here pretty big and thus less interesting to read. we moved it back at the end of the release note for this release only.
Also note that some of the changes made in this beta release will be brought in the next v3.30.0 release. They also will be documented in the corresponding release note.

Note: As this is not a stable release, installing/upgrading to it must generally be done explicity, either by adding the rx-player@v4.0.0-beta.0 package through your package manager, by adding it through its next tag (which will point toward the last beta release), by explicitly specificying its version on your package.json file(s), or through other related techniques.

A new player state: "FREEZING"

A new player state (gettable either through the playerStateChange event or the getPlayerState method) has been added: "FREEZING".

It's important to note that this new new state does not characterize a new behavior, it only put a better word onto a specific problematic playback state. Previously, it was either reported as a "BUFFERING", "SEEKING", or even "PLAYING" state depending on the situation.

This state appears when playback is temporarily stuck in place, though in opposition to the more usual "BUFFERING" and "SEEKING" state, this is not due to buffer starvation but to another, generally unknown, factor. As such, it can in most cases be handled just like a "BUFFERING" state in your application (e.g. by displaying a spinner on top of the media element) but it may be logged differently to help you pinpoint playback issues.

Screenshot: A "FREEZING" state can be treated just like a "BUFFERING" one, here with a pink spinner on top of the video.

Under that state, which is generally much rarer than a "BUFFERING" state for example, the RxPlayer will try various tricks to try to un-freeze playback. If they become too frequent or if those tricks don't work, it might be an issue worth investigating.

A more flexible track API

One of the focus of this new major release was to improve the RxPlayer API on DASH multi-Period contents - which are contents with various set of tracks and Representations (qualities) depending on the time period.

The RxPlayer's previous track API (e.g. setAudioTrack and getAvailableAudioTracks) only allowed to get the list and update the track for the currently-playing Period.

// Example setting the first english audio track for the current Period if found
const availableAudioTracks = rxPlayer.getAvailableAudioTracks();
const englishAudioTrack = availableAudioTracks.find((track) => {
  return track.language === "eng";
});

if (englishAudioTrack !== undefined) {
  rxPlayer.setAudioTrack(englishAudioTrack.id);
}

The new track API, which is still compatible with the old one (meaning the old calls still work), now allows to set the track for any Period:

// Example setting a french audio track for the first Period if found
// and an english track for the second (curious use case, but why not?)
const availablePeriods = rxPlayer.getAvailablePeriods();
if (availablePeriods.length >= 2) {
  const availableAudioTracks1 = rxPlayer.getAvailableAudioTracks(
    availablePeriods[0].id
  );
  const frenchAudioTrack1 = availableAudioTracks.find((track) => {
    return track.language === "fra";
  });

  if (englishAudioTrack1 !== undefined) {
    rxPlayer.setAudioTrack({
      trackId: englishAudioTrack1.id,
      periodId: availablePeriods[0].id,

      // Note: it's now also possible to set the switching mode per-track change
      switchingMode: "direct",

      // It's also possible to only authorize some Representations from being
      // played
      lockedRepresentations: englishAudioTrack1.representations
        .filter((representation) => {
          // Only authorize Dolby Digital+ Representations from this track
          return representation.codec === "ec-3";
        });
    });
  }

  const availableAudioTracks2 = rxPlayer.getAvailableAudioTracks(
    availablePeriods[1].id
  );
  const englishAudioTrack2 = availableAudioTracks.find((track) => {
    return track.language === "eng";
  });

  if (frenchAudioTrack1 !== undefined) {
    rxPlayer.setAudioTrack({
      trackId: frenchAudioTrack1.id,
      periodId: availablePeriods[1].id,
    });
  }
}

As you can see, the newer syntax allows more flexibility than the previous one. However it should be noted that the old syntax still work and still update the track for the currently-playing Period.

Removal of the track preferences API

When associated with new RxPlayer events also added in this version, the new track API make the preferences API re-implementable in its entirety by an application.

Because of this, and after much thinking on our side, we decided to remove the track preference API from our v4.0.0 even if we understand that it implies some work on your side to make the switch.

Don't worry the new track API is both easy to understand and more flexible than the set of methods and options that were part of the track preferences API. We also made a complete preference migration guide here, which includes code to completely replace the old preference API.

Do not hesitate to open an issue if you find that documentation not too clear or even if you're not OK with the removal of the old track preference API.
To be perfectly honest, we're still hesitating and we may bring it back in the future if it proves to have clear advantages. But for now, the new track API just seems more complete and easier to understand for applications (we had in the past several issues from people poorly understanding the behavior of the preferences API), which is why we're orienting you towards it.

Improved Representation selection

Previous RxPlayer versions only allow...

v3.29.0

Release v3.29.0 (2022-11-16)

• 🔍 Overview
• 📑 Changelog
• Configurable timeout values for the manifest and segment requests
• DRM: Advanced key expiration handling
• Deprecation of keySystems[].throwOnLicenseExpiration option
• Better URL priorization algorithm for multi-CDN contents
• Better device compatibility
• KEY_STATUS_CHANGE_ERROR Improvements
• Consideration of reverse playback
• v4.0.0 in late alpha
• Experimental "local" transport updates

⚠️ If you're reading this from the global "releases" page on GitHub, it seems that long-ish release notes, like those we like to write, are now truncated by default.
To be able to read the full one, you may need to click on the release title to be redirected to the complete content of this release note - which I find is not something made too clear. Sorry about that :/.
I also consequently decided to move the changelog closer to the beginning of that release note, so the shortened yet complete list of features, fixes and improvements become directly visible on the "releases" page.

🔍 Overview

It is time for a new RxPlayer release, the v3.29.0. It contains among other things:

• the possibility to change the used timeout for manifest and segment HTTP requests
• A new option to configure the behavior the RxPlayer will have on decryption key expiration
• A better URL prioritization algorithm for DASH contents listing multiple URL for segments
• Fixes of multiple Directfile issues
• A lot compatibility improvements, especially for LG and Samsung TVs
• Reverse playback through performing frequent small seek is now better considered by the RxPlayer (see below)

You can look at the complete changelog of this release just below, then this release note will focus on some of its main features, fixes and improvements.

📑 Changelog

Features

• add networkConfig.segmentRequestTimeout and networkConfig.manifestRequestTimeout options to loadVideo to configure the timeout of respectively segment and manifest requests [#1156]
• add timeout property to the first argument communicated to a segmentLoader (from loadVideo's transportOptions) [#1156]
• add timeout property to a new third argument communicated to a manifestLoader (from loadVideo's transportOptions) [#1156]
• DRM: add keySystems[].onKeyExpiration to loadVideo options to configure the behavior the RxPlayer should have on key expiration [#1157]
• DRM: add keyStatuses property to an EncryptedMediaError with the KEY_STATUS_CHANGE_ERROR code to communicate which key id and key status caused issues. [#1157]

Deprecated

• DRM: Deprecate keySystems[].throwOnLicenseExpiration loadVideo option as this boolean can be replaced with more customizability by the new keySystems[].onKeyExpiration loadVideo option [#1157]

Bug fixes

• Directfile: Fix long-running issues with rare "directfile" contents and some browsers/platforms (seen on Chrome PC and PlayStation 5) where playback would stay in LOADING state indefinitely despite playing [#1174]
• DRM: Fix undocumented keySystems[].videoRobustnesses loadVideo option. audioRobustnesses was previously used even for video capabilities [#1171]
• Compat/Directfile: Fix an issue with WebOS (LG TVs) when playing multiple directfile contents with the stopAtEnd player option set to true [#1154]
• Compat/DRM: Fix infinite loading on WebOS (LG TVs) 2021 and 2022 when loading more than once an encrypted content by resetting decryption capabilities each time [#1175]
• Compat: To work around an issue on WebOS (LG TVs), also specify a request timeout manually through a setTimeout call when XMLHttpRequests are created for Manifest and segment requests [#1152]
• Compat/Directfile: Fix an issue on Tizen (Samsung TVs) where playing directfile contents could randomly lead to not having audio [#1170]
• Compat: Fix issue with Tizen (Samsung TVs) where starting playback on a discontinuity could lead to infinite rebuffering [#1140, #1176]
• Compat/Directfile: For "directfile" contents, also consider AudioTrack with a description (without an "s") as audio-description audio tracks to work-around what seems to be a Safari typo [#1160]
• DRM: When using persistent licenses, create new MediaKeySession when load resolves with false, instead of relying the same, to fix issues with such persistent sessions if the browser cleaned it up [#1139]
• Only call "MediaSource.endOfStream" once, the most visible side-effect should have been repeated logs [#1163]

Other improvements

• DASH: Improve multi-CDN configurations, by smartly selecting the right CDN depending on past status [#1165]
• Allow reverse playback use cases by not skipping gaps and most discontinuities when the playback rate has been set to 0 or a negative value [#1138]
• In the experimental "local" transport, add incomingRanges property to signal the time ranges of remaining data, allowing better discontinuity handling and duration estimates for sill-loading dowloaded contents [#1151]
• Only send, through "warning" events, one EncryptedMediaError with a KEY_STATUS_CHANGE_ERROR code when multiple ones arises at the same time [#1157]

Configurable timeout values for the manifest and segment requests

Request timeouts

The RxPlayer almost continuously download media video and audio data called "segments", each with its own HTTP request, which will be then communicated to the browser so playback can start quickly, even if the full content is not yet loaded.
If one of those requests takes too much time to respond, we risk to enter a phase where playback cannot continue because we're awaiting on that data. This type of situation is called "rebuffering".

Sometimes the reason behind a request hanging for too long was just a temporary server-side issue, that may not be present anymore if the request is tried again. For those cases, the RxPlayer almost always set a timeout on Manifest and segment HTTP(S) requests it performs.

output_timeout_graphs.mp4

Video: To the top left, visualizations of the audio and video buffer, to the bottom left, current amount of buffer ahead: if it comes close to 0, it's rebuffering time during which playback is paused.
To the right, requests as seen in the Chrome inspector. We can see that two media segments called seg_16.mp4 (which are the next video and audio segments to download) each take a long time to respond. The first of these requests is then set in red and then retried: this is the timeout mechanism.
We can see the buffer quickly diving to 0 as the request hangs while playback continued.

But because we wanted to avoid as much as possible the possibility of aborting requests that are long for legitimate reasons (large data being downloaded, server processing, server-originated delay for a real reason etc.), that timeout has always been set to a relatively high value: 30 seconds.

For most contents, 30 seconds during which a particular request hangs lead to a very high chance of encountering rebuffering phase (as the pre-buffered media data is at most also 30 seconds by default), thus this made this limit not optimal.
Thankfully, requests hanging that way is relatively rare, so though we knew we could improve this situation, it hasn't been treated as a high priority until now.

Encountered issue

We recently worked on an issue partners were having where we would sometimes encounter hanging requests due to what seemed server-side issues. This in turn led to long rebuffering phases which are very unpleasant for a user.

output_timeout.mp4

Video: In this example the two media segments called seg_103.mp4 (which are the next video and audio segments to download) each take a long time to respond, with again, a timeout.
Here we can visualize that the player had to pause because we've reached the end of the buffer while waiting that next segment.

Though it may indicate that the original problem is more on the CDN side, we considered that the RxPlayer had still to provide a better experience in that case.

Moreover, resources requested in that case where small: media segments, generally the larger files being requested, were very short both in terms of size than in terms of duration.
We consequently thought about a potential improvement: if we lowered the timeout to a much shorter value (in our case 8 seconds), there may be much less chance of rebuffering in cases when requests hang. Moreover, even if we actually had a rebuffering phase, it would be much shorter in time as the request would be retried much more quickly.

The timeout API

The v3.29.0 now allows to set a tim...

v3.28.0

Release v3.28.0 (2022-07-12)

• Overview
• Changelog
• Add a label property to audio, video, and text tracks
• Adaptive content duration based on the current tracks
• DRM: Better handling of key ids absent from the Manifest
• RELOADING is now generally triggered asynchronously
• Detection of full segment Garbage Collection
• Better debugging experience and testing of complex streams

Overview

The v3.28.0 release has now been published to npm.
Though it is technically a minor release, in terms of semantic versioning, it only have few new features and mostly brings bug fixes and improvements:

• Tracks API now also optionally have a label string attribute if it found one in the Manifest
• A content's duration may now evolve in function of the current tracks chosen, to give a more precize one
• The RELOADING state is now triggered mostly asynchronously to facilitate its handling
• Media data completely garbage collected by the browser is now properly detected to avoid remaining very rare request loops
• TTML: more EBU-TT subtitles are now parsed, by considering the potential tt XML namespace we may find in those files.
• DASH: multiple small fixes have been added to raise compatibility with some peculiar MPD
• An issue has been fixed which made a set maxVideoBufferSize appear to be inaccurate in some situations.
• DRM: If the key id is not found in the Manifest file, it may be parsed directly from the media data itself, to unlock advanced features with such contents.
• Multiple other fixes and improvements (you can find the changelog at the bottom of this release note).

Changelog

Features

• Add label to audio, video and text track APIs (such as getAvailableAudioTracks) which gives a human-readable description of the corresponding track, if available in the Manifest [#1105, #1109]
• Automatically set the LogLevel to "DEBUG" if a global __RX_PLAYER_DEBUG_MODE__ constant is set to true, to simplify debugging [#1115]

Bug fixes

• Use the first compatible codec of the current AdaptationSet when creating a SourceBuffer [#1094]
• DASH/DRM: Fix potential infinite rebuffering when a KID is not announced in the MPD [#1113]
• DRM: Fix quality fallback when loading a content whose license has been cached under an extended singleLicensePer setting and when starting (and staying) with a quality whose key id is not in it [#1133]
• DASH: Avoid infinite loop due to rounding errors while parsing multi-Periods MPDs [#1111, #1110]
• After a RELOADING state, stay in PAUSED if the media element was paused synchronously before the side-effect which triggered the reloading (usually coming from the API) was perform [#1132]
• Fix issue with maxVideoBufferSize setting which could lead to too much data being buffered [#1125]
• Prevent possibility of requests loops and infinite rebuffering when a pushed segment is always completely and immediately garbage collected by the browser [#1123]
• DASH: Fix potential rare memory leak when stopping the content after it has reloaded at least once [#1135]
• Directfile: Properly announce the audio track's audioDescription accessibility attribute in directfile mode on Safari [#1136]
• DASH: Fix issues that could arise if a segment is calculated to start at a negative position [#1122]
• DASH: Fix possibility of wrong segments being requested when a SegmentTimeline in a given Period (whose Period@end is set) had an S@r set to -1 at its end [#1098]
• DASH: If the first <S> has its S@t attribute not set, make as if it is set to 0 [#1118]

Other improvements

• TTML: Add support for percent based thickness for textOutline in TTML Subtitles [#1108]
• If seeking after the last potential position, load last segments before ending [#1097]
• Improve TypeScript's language servers auto import feature with the RxPlayer by better redirecting to the exported type [#1126]
• The duration set on the media element is now only relative to the current chosen tracks (it was previously relative to all potential track). This allows to seek later when switching e.g. to a longer video track [#1102]
• Errors coming from an HTMLMediaElement now have the browser's error message if it exists [#1112]
• TTML: Better handle EBU-TT subtitles by handling the tt XML namespace in our TTML parser [#1131]
• DRM: Information on persisted DRM sessions are now automatically updated to their last version when possible [#1096]
• Only log values which are relatively inexpensive to stringify to reduce the difference between debugging sessions and what is usually seen in production [#1116]

Add a label property to audio, video, and text tracks

Current track labeling

To choose between several audio, video or text tracks, an application had previously only access to its key characteristics: its language (for audio and text tracks), codecs, accessibility features, resolution of its qualities (for video tracks) etc.

If they had to display a track choice to the user, applications usually then derived a name from these characteristics.
For example, french closed captions could be proposed as "French [CC]", a dolby digital + italian audio track as "Itialian [Dolby Digital +]", and so on.

Screenshot: one of the user interfaces used at Canal+ for french users. The various audio and text tracks' names proposed there are actually entirely derived from the tracks main characteristics such as the language and accessibility features.

All this works pretty well but this system have some limitations. For example, different tracks could have the same exposed characteristics in which case they would have the same name.
Moreover, it severely limits the information you can give to the final user to only those exposed characteristics. What if you want to indicate that an audio track of some sport game only contain stadium noise for example?

The DASH <Label> element

The DASH-IF IOP, a DASH standard we follow, comes to the rescue by authorizing setting a Label element on the corresponding <AdaptationSet>, which contains a description of what the track is about. This is explicitely done to provide a description of the track to the user in the final user interface.

Example of Label elements on multiple text AdaptationSets, generally signalling how the track choice should be worded.

This is not limited to DASH either. For example HLS also has a NAME attribute for roughly the same thing (though only DASH's Label is handled for now).

In the RxPlayer API

Now, if <Label> elements are present in a DASH MPD, the RxPlayer will provide a label property through its various tracks API:

• In track objects of the getAvailableAudioTracks, getAvailableVideoTracks and getAvailableTextTracks methods, to know the label of all the current track choices

• As property of the object that may be returned by the getVideoTrack, getAudioTrack and getTextTrack method, to know the label of the currently-chosen track

• As property of track objects of the linked availableAudioTracksChange, availableTextTracksChange, availableVideoTracksChange, audioTrackChange, textTrackChange and videoTrackChange events.

Output of a getAvailableTextTracks call in the Chrome inspector's console after playing the content linked to the previously's pictured MPD

In cases where no <Label> element is present on the corresponding <AdaptationSet>, the label property may either not be defined on the track objects or set to undefined.

As any DASH feature, this is available both with our default JavaScript DASH parser and with our WebAssembl...

v3.27.0

Release v3.27.0 (2022-03-31)

The v3.27.0 release has now been published to npm.
It is a big release which includes many features, fixes and improvements, among which:

• A new option, maxVideoBufferSize, allows to indicate to the RxPlayer that the current device has limited available memory.
  The RxPlayer will then use that value to prevent overflowing it by loading video segments adaptively

• A new audioTrackSwitchingMode, "reload", allows to work-around some compatibility issues seen on some recent Chrome versions, where audio could be lost after switching audio track in the "direct" mode

• Safari support has been improved for some advanced usages

• DRM: the keySystems[].singleLicensePer option now accepts a third license requesting mode: "periods". It enables optimizations on use cases like key rotation and support of targets with a very limited number of key slots available.

• DRM: the "content" mode of the keySystems[].singleLicensePer option is now more optimized: persistent and non-persistent caches should now lead to more cache hit than before under this mode.

• Many other bug fixes and improvements, you can refer to the changelog for more information

Configuring a maximum video buffer size

Issues with low-end devices

At Canal+, we use the RxPlayer in production on a large panel of devices. Some of them being real powerhouses without any performance or memory issue, others being more limited on processing speed and memory.
When playing high-bitrate media contents - such as with ultra-high video definition (UHD) - on some devices of that second category, we sometimes entered a situation where the memory of the device would quickly become full as video content was buffered by the RxPlayer, leading to all kinds of performance and media issues.

The situation was particularly problematic on some older smart TVs, where we had temporarily advised application developpers that they should set for now a lower wantedBufferAhead.

This configuration option configures the amount of media content the RxPlayer will pre-load in advance. Setting a lower value thus reduces the amount of loaded media data and thus lowers memory usage, but it also raises the risk of rebuffering.

Image: Difference between a low wantedBufferAhead (left) and a high one (right), we can see that we load more segments in the buffer when that value is high (visible both in the buffered grey part of the progress bar or in the visual representation of the audio and video buffer's content at the bottom - don't mind the color used there, they are randomized)

Moreover, it was a general setting: lower-bitrate video or even audio would also respect this value - even if only high-bitrate video was problematic.
We thus decided to provide a proper answer to this specific problem.

The maxVideoBufferSize solution

The solution we came with was to add a maxVideoBufferSize option to the RxPlayer.

The idea is that the application developers might know in some cases much better than the RxPlayer the limitations of a specific device their application runs on.
Setting a maxVideoBufferSize will thus serve as a hint telling the RxPlayer how much video data, in kilobytes, it should buffer at maximum to prevent most of those memory-related issues.

The RxPlayer will then use this hint to fill the video buffer adaptively:

• while video data is still below this maxVideoBufferSize limit, the RxPlayer will try to reach the wantedBufferAhead setting, as usual
• as soon as this memory limit is encountered, the RxPlayer will in most cases stop loading new video data until that limit can be respected again - this happens for example once old video data is garbage-collected.

To implement this feature, the RxPlayer now estimates the storage size used by video media segments present in the browser's buffer as well as of future video segments it wants to download. Additionning those allows to check if the maxVideoBufferSize limit is still respected.

Image: Here we're playing the same content with equal wantedBufferAhead and maxVideoBufferSize (10 Megabytes) but different qualities. To the left is a video of a lower quality, to the right a higher one. We can see that the buffer size constructed on the right is much lower. This is here because the maxVideoBufferSize limit was reached. We can also see that on the left, the constructed buffer is roughly equal between audio and video, this indicates that the maxVideoBufferSize was probably never reached before the wantedBufferAhead.

However that type of estimate is not an exact science, the real memory reserved by the browser to handle those segments might actually be higher or lower. The RxPlayer may also decide by itself to still load some video data once that limit is reached if it judges that the buffer built is too low for smooth playback.
As such, this maxVideoBufferSize option only acts as a hint and not a true hard limit.

The maxVideoBufferSize API

This maxVideoBufferSize limit can be set at either one of two places.

Either in the constructor, documented in the constructor options page:

const rxPlayer = new RxPlayer({
  videoElement,
  /**
   * This is expressed in kilobytes, so here this corresponds to 50MB or
   * ~47.68 MiB.
   */
  maxVideoBufferSize: 50000,
});

Or at any time (even as content plays) through a method:

rxPlayer.setMaxVideoBufferSize(50000);

This method's documentation is under this link.

The last set maxVideoBufferSize can be retrieved through the following getter:

rxPlayer.getMaxVideoBufferSize();

This method's documentation is under this link.

Note that by default this value is set to Infinity, which is also the value to set it to to disable this limit.

DRM: A "periods" mode for the keySystems[].singleLicensePer loadVideo option

The singleLicensePer: "content" mode

Last year (v3.24.0), we added the singleLicensePer property to our DRM-related options.
It was added to optimize the playback of contents with multiple decryption keys: by only performing a single license request for all decryption keys instead of the usual one per key, we were able to reduce the time and resources needed to play those contents both on the client and on the server-side.

Image: By default, the RxPlayer will load a single license per new encryption metadata encountered. This is adapted when the license server emits only the key asked, like illustrated here.

Image: Through the singleLicensePer: "content" mode we implemented at the time, the RxPlayer will load a single license for the whole content instead. This only works if the license server returns all keys linked to the content at once, and not just the asked one.

Technically, this option changes the RxPlayer's behavior on two aspects:

1. Only a single HTTP(S) request is performed - even when multiple keys are needed concurrently
2. It also communicates to the RxPlayer the fact that any key that is not found in the license - yet should have been in it - are keys voluntarily witheld by the license server.
   Thus the Representation (a.k.a. profile or quality) corresponding to those witheld keys should not be played (and be fallbacked from if already buffered).

Image: In this example, the license request for the video 1080p did not return a key for the 4K video quality. Because we're in a singleLicensePer: "content" mode and thus should have received all available keys, the RxPlayer deduce that the 4K video quality is not decipherable. It will thus avoid this quality and remove it from the video buffer if it was already pushed in it.

This option is used a lot at Canal+, where the majority of customer-facing applications use that "content" mode, so we can play contents faster and put less stress on our back-end.

Inconvenients of this mode

However, performing only a single license request for the whole content has some drawbacks, the main ones being:

• All keys needed to decrypt any part of the content has to be known in advance, even decryption keys for future sub-contents, because no new license request will be performed in the future (all those keys we didn't know of in the first license request will be assumed to have been voluntarily witheld).

• Some contents could need a lot of decryption keys and some peculiar devices have a relatively low limit of "key slots" available.
  In that case, it might be problematic to push a single license containing more keys than what the device can handle, leading to all kinds of playback errors.

  

  _Image: Case of a device with only 3 "key slots" available. Here, if receiving a license containing 4 keys, the device would not be able to use...

v3.26.2

Release v3.26.2 (2022-01-11)

The v3.26.2 release has now been published.

It includes multiple bug fixes and improvements:

• The RxPlayer has now a better adaptive logic when playing DASH low-latency contents. This is explained with details below.
• Low-latency contents using a $Time$ indexing scheme, are now better handled
• The RxPlayer's documentation pages have been updated in depth, now (finally) including search capabilities!
• The player is now more resilient when parsing an MPD with unreachable optional resources, such as the ones behind <UTCTiming> elements
• With DASH contents, <ContentProtection> elements can now be declared at the <Representation> level
• many other small bug fixes and improvements (see the changelog at the bottom of this release note).

We'll now go in more details on some of those improvements.

New adaptive algorithm for DASH Low Latency contents

The low-latency adaptive status

It has now been more than two years since the RxPlayer introduced the lowLatencyMode option, allowing to play efficiently DASH Low Latency contents (often called DASH-LL or LL-DASH).

output.mp4

video: Example of playing the same DASH-LL content without lowLatencyMode (to the left) and with it (to the right). We can see that the right one plays almost ten seconds forward.

However there was one area where we knew we could do much better: the selection of the quality to play a.k.a. the adaptive logic.

The adaptive logic is notoriously an important issue when it comes to playing low-latency contents, due to specificies explained below.
When we started to implement low-latency playback, we tried to devise an algorithm specifically for it, but we didn't get good-enough result for us to go on with it.

What we decided instead was to keep our current less-than-ideal adaptive algorithm (which just risk to play a lower quality than what could be expected) and mostly wait for a more suitable adaptive algorithm to make its apparition. We felt that it was now the right time!

Problems with the current adaptive algorithms

The current adaptive algorithms

Previously, the RxPlayer relied on two kinds of algorithms for its adaptive logic:

• bandwidth-based algorithms: where we measure the current user's bandwidth by observing how fast media segments are downloading and choose a media quality with the highest bitrate compatible with that bandwidth.

  
  Depiction of the content of the video buffer when running the RxPlayer's bandwidth-based algorithm only.
  The blue color corresponds to the initial quality (which is lower than ideal), yellow is the ideal one and the red bar corresponds to the current playing position. Here we can see it reaches the ideal quality after some time loading the initial one. This is because it first had to perform bandwidth calculation with that initial quality before settling on the yellow.

• buffer-based algorithms: where we choose a quality depending on how much data has already been buffered in advance.
  If we have not much data in the buffer, we switch to a low quality to fill it, if however we have a lot of data in advance in the buffer, we can switch to a higher quality without much risks of rebuffering.

  
  Depiction of the content of the video buffer when running the RxPlayer's buffer-based algorithm only.
  The blue color corresponds to the initial quality (which is lower than ideal), yellow is the ideal one, green is better than the ideal one and the red bar still corresponds to the current playing position. Here we can see it reaches the ideal quality after more time than with the bandwidth-based approach but also reaches a better-than ideal quality at the end of the buffer, because the buffer has been filled.

Both are widely used type of algorithms which have advantages and disadvantages:

• bandwidth-based algorithms are often very fast to settle on a good-enough approximate of the ideal quality, yet in its current form are often set to be a little too pessimistic to make-up for potential network latencies and bandwidth variations.

• buffer-based algorithms allows to reach a higher quality as long as enough data is buffered. Sometimes even better quality than what the current network connection would allow (when enough buffer has been built).
  Yet it is slow to start (as data has to be buffered initially) and needs the ability to pre-buffer data, which is problematic for live contents because there the player plays too close to the live edge to be able to pre-load much of it.

Which is why in normal playback (non-low-latency ones), the RxPlayer will use a mix of the two: in most cases it will consider the best quality between what both kinds of algorithm estimate.

Issues with DASH low-latency contents

Sadly both buffer-based and network-based approachs break down when playing DASH Low Latency contents:

One of the principles behind DASH Low Latency is that small chunks of media data, called segments, can be requested while it is still generated in the back-end.
When that is the case, the request for a segment stays open until it is fully generated and transfered. In that situation, it's difficult to rely on the time it took to perform the request (as we do to perform bandwidth calculations), because a large part of that time might just have been spent "waiting" for the segment to be generated.

Also, the main point of low-latency being to reduce the delay between video/audio capture and its presentation to the user, the RxPlayer will play too close to the live edge to build enough buffer for buffer-based algorithms to be useful.

Depiction of a video buffer playing too close to the live edge (the red bar is the current position, the end of the container represents the current live position) to be able to build much buffer for the buffer-based approach

So none of the two previous approaches really worked, leading to a poorer quality thans what could be expected.

Algorithm specific for low-latency contents

As we were not in a hurry, our initial strategy was consequently to wait and see until the technology was more mature.
Some months ago, we noticed that multiple new published algorithms could finally be used for low-latency content specifically, some even having already an implementation in the open-source dash.js media player.

We first tried to implement one of those algorithms, called Learn2Adapt. Yet the implementation was less than ideal due to the difficulty to provide one of its input. When we checked dash.js, which can also rely on this algorithm for low-latency contents, that algorithm was also not satisfying.

After this failure, we redoubled our efforts and we worked on multiple fronts:

• We tried to implement another algorithm from scratch.
  It was written as a temporary solution which was better than what we had today, but could be less optimal than another, low-latency specific, one.

• we worked on a separate tool, whose goal was to test our adaptive algorithm in various network conditions (poor bandwidth, high latency, variable conditions etc.) and compare it to other media players (mostly dash.js and the shaka-player).
  

  Example of a graph outputed by this tool. Between other things, it allows to quickly check the evolution of the video bitrate and of the size of the buffer

Thanks to this approach, we were able to develop a new algorithm with confidence.

That new algorithm was called the "guess-based [quality] chooser".
It may sound scary to have the word "guess" in there, but it translates well what this algorithm is doing: trying, gradually, to guess what the ideal quality is.

The guess-based chooser

The GuessedBasedChooser is actually a class in the RxPlayer that implements the corresponding "guess-based" algorithm.

It relies, among other inputs, on a core "maintainability score" concept linked to each quality.
This score indicates if we seem to be loading that quality faster or slower that the time it takes to play it:

• if we load it faster, this quality can be maintained without the RxPlayer rebuffering. In that case the quality will have a high maintainability score.
• if we load it slower than the time it takes to play it, the RxPlayer has a high risk of rebuffering (at some point, we'll have to wait for data to load). Here the quality will have a low maintainability score.

The idea is then to use that score (and other inputs), to know if a quality is maintainable or not, and take the corresponding actions depending on that:

• if it seems maintainable enough for a long time, we may try the immediately superior quality, we could say the RxPlayer is "guessing" that the immediately-superior quality might be maintainable.
• it it doesn't seem maintainable (even for a very short time), we might go the other way: switch to the immediately inferior quality. We call that situation "taking a wrong guess", as the current quality was not a "good" (maintainable) guess.

  
  _Example of a wrong guess. The RxPlayer was initially playing the mainta...

v3.26.1

Release v3.26.1 (2021-09-14)

The v3.26.1 release has now been published.

It is a relatively small release which provides minor improvements and works around rare browser issues:

• We encountered very rare situations where the player would indefinitely freeze due to some contents and browser issues.
  We now work-around those by automatically "unfreezing" the player when it froze for a given time.
• Likewise a subtle browser issue could in rare scenario lead to media data being requested multiple times in a loop.
  The player now detects this situation and prevent that loop from happening
• We greatly improved the DASH_WASM experimental feature (which accelerates and improves the memory situation when parsing large DASH MPD by relying on a WebAssembly parser).
  Mostly its size and the time to compile it have been drastically reduced.
• More types are exported through the rx-player/types path.
  Those were types needed from already exported APIs, such as types about track preferences.
• Infinite rebuffering issues that could happen when some Periods in a multi-Period contents had no video content should now be fixed
• We continued our efforts to improve on initial loading time, by requesting what is called the initialization segment parallely to the first media segments when it makes sense.

As those are mostly very technical improvements, we will try to explain and dive into the details of what this release brings in the next chapters.
If this is too much details, you can jump as always to the changelog at the end of this release note!

Better handling of "frozen" playback

This release works around a very rare issue we infrequently encountered where the content appears to be indefinitely rebuffering.
This case is internally called "freezing".

How content playback basically works with the RxPlayer

To introduce what we call "frozen" playback, we need to introduce the basics of how content playback with the RxPlayer works.

First, the RxPlayer does not decode audio and video data itself, it relies on the browser (and underlying codecs) to do so.
What happens is that the player fetches the right audio and video "segments" (small decodable parts of the content) and indicates to the browser the position that should be played.
The browser and the codecs it integrates then ensure that the audio and video content corresponding to that position is decoded.

To ensure smooth playback, the player has to make sure that audio and video segments are "pushed" to the browser before the decoding operation reached their corresponding starting positions (or said another way: before we need one of them).
If the player couldn't load the segment before that deadline, the browser won't have anything to decode and will sort-of "freeze" until that data is pushed.

That behavior is also called "rebuffering" or more colloquially just "buffering".

Example of rebuffering. We can see, below the video, a Buffer content chart. It indicates that the current position (the red line in both buffers) is close to the end of the currently buffered audio and video data, we're thus awaiting new data before being able to play

"Freezing"

Yet in rare circumstances, that freezing situation can happen even when the right segments have been "pushed".
There can be legitimate reasons, like when a needed decryption key has not been obtained yet. Though there are times when there seems to be no apparent cause, at least from the point of view of the player.

In the RxPlayer team, we call that problematic case "freezing" (whereas regular rebuffering cases are just internally called "rebuffering").
Thankfully, this happens in very rare circumstances. We usually encounters unexplained stucked playback only in the following cases:

• poorly-packaged contents
• playing with a very high playback rate (i.e. at a high speed)
• when the player is doing risky behavior in some particular platforms - for example pushing a video segment containing data for the already-buffered current position on some old samsung TVs could lead to that situation.
  We however worked hard on this last category to avoid them in most cases.

Here we can see, that the content seems to be rebuffering despite having both audio and video data in front of the current position (the red lines in the Buffer content chart). Here this actually happened after I set a playback rate of 10 (meaning I'm playing the content at 10 times its regular speed) and waiting.

Even if very rare, it can happen and having a logic in last-resort would not be superfluous.

Going out of that frozen state back into regular playback is usually pretty easy.
In most scenarios, a simple little "seek" of 1 millisecond (which means going 1 millisecond in the past or in the future) is sufficient to "unfreeze" playback.
This is probably the case because seeking re-triggers some lower-level code which was linked to the source of freezing, e.g. low-level audio and video buffers could be emptied and filled again and the lower-level playback logic could be restarted from the new "seeked" position.

The new logic

So in definitive, the resolution of that issue is simple: if the player detects that "freezing" is happening, we just have to perform a little seek to unfreeze.

However, detecting when the player is abnormally frozen is not really straightforward.

The naive way would be just to look at two different point in time when playing with enough data, and see if the current position has evolved. If it did not, it must mean that we're stuck and - because we have data - playback must be freezing.

Yet there are multiple issues with doing just that:

1. Poor performance could lead to false positives.
   For example, we noticed that after a seek, some low-end devices can take up to multiple seconds to restart playback, even when enough data is available to play.
   By resolving freezes the naive way, the player thus might think it is frozen because a seek operation is taking a long time to finish. Here it might perform another seek, which will lengthen even more playback (we will also risk to be in an infinite seeking loop !).
2. There are sometimes good reasons for playback not restarting yet. The main one being that the decryption key for the incoming data is not yet obtained.

When considering all possibilities, this "freezing" state thus becomes very complicated to detect.
We ended-up relying on an imperfect yet sufficient solution:

• The player only performs the seek if playback is abnormally stuck after a very high delay (between 6 and 7 seconds).
  Being frozen for around 7 seconds sounds like a lot, but this logic should only run as a last resort anyway.
• A lot more playback characteristics than just the current position are looked at to deduce if we should be able to play.
• Already-existing and less risky "unfreezing" strategies will run before it.

On the same case of freeze while playing at a very high playback rate, we can see that this logic is finally able to un-stuck us.
Here is a case with that logic in action (with added logs):
https://user-images.githubusercontent.com/8694124/124958710-bf687e80-e01a-11eb-94a8-3c2ebaad7301.mp4

Removal of the cached segment detector

Web browsers maintain a somewhat complicated HTTP cache to avoid requesting multiple times already-fetched resources.

Thanks to it, a request made normally from JavaScript (through either an XMLHttpRequest object or a fetch call) will sometimes avoid the round-trip to the server. The main advantage of doing that is that it reduces latency.
All this happens without the knowing of the RxPlayer, for which the request still appear as if it was performed through the network.

Yet detecting when a response comes from the cache is sometimes needed by the player.
For example, the internal logic calculating the network bandwidth (to select the optimal video and audio qualities) relies heavily on the measure of the time it takes to download data from the network. When such data comes from the cache instead, it appears that the user has an exceptionally high bandwidth.

So until this release, the RxPlayer had what we called a Cached Segment Detector, whose job was to deduce heuristically whether data came from the browser cache or the network. It basically compared times to download such data and ignored all metrics which indicated an abnormally high bandwidth.

  -----------------
 | network request | 1. A network request metric is generated
 |     metric      |    after a request
  -----------------
         |                         2. "does it come from
         |                            the browser's cache?"   +----------------+
         +-> +------------------+ --------------------------> | Cached Segment |
             | network listener |               3. "yes"/"no" |    detector    |
             +------------------+ <-------------------------- +----------------+
                     | 4. If it doesn't seem to come from the cache
                     |    (and other rules)
                     |          +-------------------+
                     +--------> | Network bandwidth |
                                |     calculator    |
                                +-------------------+
                                        | 5. gives bandwidth
                                        ...

v3.26.0

Release v3.26.0 (2021-06-10)

The v3.26.0 release has now been published.

You might ask: where is the v3.25.0?
We had several issues while trying to publish it. Instead of releasing officially a v3.25.2 (as we had two issues while publishing), we guessed it might be more logical for an user as a new minor version and abandoning/deprecating the 3.25.x versions.

This release includes among other improvements:

• DASH trickmode tracks can now be handled in two ways:
  1. for trick play (e.g. playing at a faster rate a video track with a lesser framerate)
  2. for thumbnail generation (i.e. "preview thumbnails" backed by a video element, rendering frames taken from that trickmode track).

• HDR information is now parsed for DASH contents and is communicated through the corresponding video tracks APIs

• A new "experimental" (meaning stable but the API may change in future versions) DASH MPD parser, relying on WebAssembly.
  It has been seen to be 4 to 5 times faster than the regular JS MPD parser when handling very large MPDs (of several megabytes in size) based on large <SegmentTimeline> elements.

• The audioTrackSwitchingMode option of loadVideo, which allows to configure the player's behavior when changing the audio track, should now be able to handle the "direct" mode mostly without needing to reload the content.
  As it seems to provide much nicer and faster audio transitions than the default, we suggest that most applications set it (we do not set it by default because of a possibly API-breaking behavior change).

• The size of the cache of encryption sessions (call "MediaKeySession") is now completely configurable for when you know the limitations of the current device

• plain-text subtitles files (i.e. that are not embedded in a container file) are now better handled. In previous versions, relying on them could lead to the player keeping re-fetching them

DASH Trickmode handling

This new release allows to display trickmode tracks when available, anytime the application wants to.

What are trickmode tracks?

"Trickmode" tracks are specific video tracks which only show a subset of the frames of another video track.
You can see it as a copy of a video track, at a much lower frame rate.

The idea behind those is that a player can select those instead of the original one when playing in fast-forward/backward mode:

• trickmode tracks are generally in a much lower bitrate, which means less bandwidth is taken by playing those (even more when considering that the playback speed multiply that value)
• it is generally much easier to decode (even more when considering that only intra-frames are typically present), leading to less CPU usage, which could be a problem at higher speed
• it provides a different aesthetic, which might be preferred when fast-forwarding, for example

DASH contents allows to define trickmode track though its Manifest document, the MPD (as far as we know, HLS also supports those type of "I-Frames only" contents).

Preview

We made a quick video to show the result of enabling/disabling trickmode tracks on our demo page:
https://user-images.githubusercontent.com/8694124/121335185-a769e600-c91a-11eb-9bee-7bb81d727971.mp4

Here, we update the playback rate to x10 and switch to trickmode tracks, then we return to a regular speed and disable trickmode tracks.
More information on the API and behavior below.

You can notice a visible black screen when enabling/disabling the trickmode track. This is because the RxPlayer is currently switching to the "RELOADING" state when this happens.
This can be a annoyance. We plan however to remove that step in future versions.

How it is handled by the RxPlayer

As described in the previous chapter, trickmode tracks are most-of-all useful when playing a content at faster speeds.

We began its implementation by introducing new APIs and events specific to this use-case, but finally found out that it was too complex and too hard for an application to manage.
We thus ended up just relying on already-existing APIs.

Enabling / Disabling

To enable or disable trickmode tracks, a new optional object argument has been added to setPlaybackRate:

// Play at x5 the regular speed and enable trick mode tracks if available in the
// current content/Period
rxPlayer.setPlaybackRate(5, { preferTrickModeTracks: true });

// Not setting `preferTrickModeTracks` doesn't change the value it was set to
// previously.
// Here we stay with trickmode tracks enabled and just update the speed.
rxPlayer.setPlaybackRate(3);

// Play at regular speed and disable trick mode tracks
rxPlayer.setPlaybackRate(1, { preferTrickModeTracks: false });

As you can see, setting the preferTrickModeTracks property on that object allows to enable or disable trickmode tracks.
Moreover, this setting can be set when no content is playing and is persisted to the current instance of the RxPlayer (exactly like the playback rate is), so if you want to disable it when switching content, you will have to call stPlaybackRate again, with preferTrickModeTracks set to false.

You can also know at any time if that option is active through a new RxPlayer method, areTrickModeTracksEnabled:

const areEnabled = rxPlayer.areTrickModeTracksEnabled();
if (areEnabled) {
  console.log("Trickmode tracks will be used when available.");
} else {
  console.log("Trickmode tracks are not enabled.");
}

Note that for now, enabling or disabling trickmode tracks may temporarily show a black screen while the player is in a RELOADING state. We might improve on that in future releases however, to avoid reloading at all in most cases.

Events

To know WHEN a video trickmode track is enabled and to know when it is disabled, we again chose to rely on the already-existing videoTrackChange event.

On trickmode video tracks a new isTrickModeTrack property will be set to true. You can rely on it to know if the currently chosen video track is a trickmode track:

rxPlayer.addEventListener("videoTrackChange", (videoTrack) => {
  if (videoTrack.isTrickModeTrack === true) {
    console.log("Now switching to a trickmode track");
  } else {
    console.log("Now switching to a regular video track");
  }
});

Likewise, getVideoTrack will also have the same property (getAvailableVideoTracks, however, won't list trickmode tracks, as those cannot be switched to through the setVideoTrack method).

On all APIs returning video tracks (videoTrackChange/getVideoTrack/getAvailableVideoTracks) another optional property has also been added, trickModeTracks.
If set to an array, it contains the trickmode track(s) linked to this "regular" video track. You can use this for example to know when a track is linked to one or several trickmode tracks.

If no trickmode tracks are available

We chose the word prefer in preferTrickModeTracks on purpose because the played content might not have any trickmode track.

In that case, the regular video track will continue to play. This is also valid in DASH multi-Period contents where some Period do have trickmode and others do not.
Here, the trickmode track will be used when available and if not, the regular video track will be played instead.

Documentation

As usual, the documentation of the corresponding API have been updated:

• documentation for setPlaybackRate
• documentation for areTrickModeTracksEnabled
• documentation for the videoTrackChange event
• documentation for getVideoTrack
• documentation for getAvailableVideoTracks

Generating thumbnails from DASH trickmode tracks

Another possible use-case for trickmode tracks, is to use them for generating "preview thumbnails", for example while hovering the seek bar.

This release adds the experimental[1] tool, the VideoThumbnailLoader.

Here is an example of how you can use it:

import RxPlayer from "rx-player";

// import the VideoThumbnailLoader and the `DASH_LOADER` to handle DASH
// contents.
import VideoThumbnailLoader, {
  DASH_LOADER
} from "rx-player/experimental/tools/VideoThumbnailLoader";

const player = new RxPlayer({ /* some options */ });

// Link logic to handle DASH segments
VideoThumbnailLoader.addLoader(DASH_LOADER);

// Video element used to display thumbnails.
const thumbnailVideoElement = document.createElement("video");

// Link VideoThumbnailLoader to the RxPlayer instance
const videoThumbnailLoader = new VideoThumbnailLoader(
  thumbnailVideoElement,
  player
);

player.loadVideo({ /* some options */ });

// Ask for the VideoThumbnailLoader to fetch a thumbnail for the current
// content that should be displayed at presentation time = 200 seconds.
videoThumbnailLoader.setTime(200);

Because the VideoThumbnailLoader uses video segments to display thumbnails, it has to rely on an HTML video element, and not an <image> element like one would probably expect.

We tried hard to make the API as easy to use as possible, you can see its documentation here.

To show you a preview of how it can look like, here is an example now available in our demo page:
https://user-images.githubusercontent.com...

v3.24.0

Release v3.24.0 (2021-04-01)

(No, it's not an april fools joke, unless it doesn't work, in which case we will pretend it was)

A new release is here! Amongst the new features and improvements:

• a new inbandEvent event is now triggered when event metadata is encountered in a segment (for example through "emsg" boxes in an ISOBMFF segment), just before that segment is pushed

• a new singleLicensePer option now allows to optimize the loading of contents with multiple decryption keys, to be able to perform only a single license request.

• we improved a lot the compatibility of the RxPlayer with several devices, most importantly many Tizen versions (e.g. Samsung TVs) and the PlayStation 4 (as to the PlayStation 5, it should already be handled).

• multiple optimizations have been added to allow lower content loading times

• many other bug fixes and improvements (you can see the changelog at the bottom of this release note)

A new player event: inbandEvent

"Inband" events

Streaming formats specify different ways to emit events through a content.
Those events can be sent at any time when streaming the content, and can be used to signal generic or format-specific information to a player or the application (e.g. signaling about parental ratings on the current content or about the Manifest's validity).

The RxPlayer already supported one kind of event: the MPEG-DASH manifest EventStream node, allowing an event to be declared directly in the Manifest.
But instead of only having events in the manifest, those can also be found inside media segments.

MPEG-DASH uses such format, by exploiting the "emsg" box of the ISOBMFF standard. This box is included in the metadata part of ISOBMFF segments (as in: it is not included in the media data itself) and is usually related to the media presentation time.

Those type of events, which are included directly in the segments, are usually called "inband events".

In the RxPlayer: inbandEvents event

The RxPlayer now parses these boxes (only the version 0 of it for now) and :

• Either the event has a specific meaning for a player (generally to give information
 about the manifest validity, allowing the player to consider when it may refresh the manifest) in which case it will handled it by itself.
• Either, the event is generic, or specific but not yet handled by the player. In which case it will be parsed and emitted 
by the RxPlayer through an event.

This new event is called "inbandEvents".
It emits one or more parsed inband events, as several "emsg" boxes may be found when parsing a unique media segment.

The "inbandEvents" event is emitted just after the corresponding segment is parsed, unlike the already-existing "streamEvent", which is sent at the presentation time of the concerned event.

And as an "emsg" box is associated with a specific segment, the player send it at the moment it gets it, before that segment is actually pushed.
This is important as it may be used to signal that the current segment contains content that may be subject to rating restrictions.
In that case, the client may need to interrupt playback before the segment's data is buffered.

This new event and its format is documented here.

singleLicensePer option: single license request for contents with multiple keys

Previous situation

When playing encrypted contents, the RxPlayer has a simple logic to know when it needs to perform a licence request:

1. Let's say that the player switches to a new Representation/quality
2. It looks at that quality's linked encryption initialization data if it exists:
   • if it was already encountered, it does nothing in particular with it
   • if it wasn't yet encountered, it goes to the next step
3. The player creates a new decryption session (called MediaKeySession) associated to that data and ask that session to generate a license request with it
4. The browser sends the player an event back with a "challenge" allowing it to perform a license request by calling the getLicense callback
5. Your application fetches the license and give it to the RxPlayer, which then pass it down through the MediaKeySession.update API
6. The RxPlayer then looks at which keys were contained in that license and reacts accordingly (it might fallback for unsupported qualities etc.)

This works pretty well, but it generally leads to multiple license requests when a content has different keys depending on the quality chosen.

This is because Representations linked to different keys will generally have different initialization data. The RxPlayer will thus create a new MediaKeySession and perform a new license request for each of them.

Optimization: performing only a single license request

We've looked at possible ways to optimize this logic. The most evident solution is to only perform a single license request.

Yet that would mean that the license returned contains all needed keys.
As such, this optimization relies on two things:

1. the license server has to be aware of this "trick" and return all keys even when a single one is asked for
2. the RxPlayer has to only perform a single license request.
   It could still technically perform multiple ones but this would be pointless as the first one already gave everything needed.

For that second part, we added the new singleLicensePer properties to the keySystems option of the loadVideo API.
By setting it to "content" (more possible values might come in the future), you can signal that there's only a single license available for the whole content.

It can be used as such:

rxPlayer.loadVideo({
  // ...
  keySystems: [{
    // ...
    singleLicensePer: "content",
  }]
}

This option is documented inside the keySystems documentation.

Remaining problem: what about missing keys?

But if the RxPlayer only decided to perform a single license request and did not change anything else in its behavior, it might lead to an other issue: what should happen if one of the keys is not in the license?

This might happen when the CDM, the module plugged in the browser allowing content decryption, is not trusted enough by the license server, resulting in the latter withholding some keys (the more restrictive ones).

The decision we've made was that, in the case an awaited decryption key isn't present in the pushed license, the player will automatically decide to mark the corresponding Representations as "undecipherable".

In this situation, those qualities won't be played. If segments corresponding to that quality were already pushed (we load media segments in parallel of the license request as an optimization measure), the RxPlayer will first try to remove them and might in some very rare conditions switch to the "RELOADING"state while doing so.

Improved compatibility with several platforms

At Canal+ Group, we recently had a wave of new platforms on which the RxPlayer was ported (I'll just name-drop the PlayStation 5, PlayStation 4 and Samsung TVs but there's others).
This was the occasion to improve our records on portability.

The main issues we encounter when porting the RxPlayer are subtle browser behavior differences, sometimes still specification-compliant and sometimes not, most of them being related to their respective MSE/EME implementations.

There was a lot of changes on our side (and multiple times on the implementer's side! We don't always want to fix another application's mess :p).

Most notably, they're fixing:

• A mysterious infinite loading situation when beginning a live content on some platforms (amongst which, Samsung TVs), most likely due to an overflow on the browser-side after setting the duration (#914)
• A situation where both the browser and the RxPlayer wanted to seek at different times, leading to a conflict (#922)
• decoding issues we encountered when pushing segments on top of the current position (#925)
• A leak of MediaKeySession on platforms where the generateRequest API is especially long (#920)
• Persistent MediaKeySessions being thrown away because too much time passed before the player was aware of its linked keys (#928, #926)

Loading time improvements

In this release, we also began (and still continuing) trying to heavily optimize the initial content loading time:

• One of the main improvements is that now when calling loadVideo when a content was already playing, the new content's manifest will be loaded in parallel of the old content being stopped (implementation details: #894)

• Another one, only for encrypted contents, is to depend only on the encryption initialization data in the Manifest if it already contains the right one (instead of the default behavior, which waits for a segment to be fetched first).

  This optimization was a complex one to implement because it implied recognizing which encryption initialization data is needed, whereas the previous implementation just used all available data - a solution which always work (implementation details: #911, #919).

  If you rely on persistent licenses, this optimization won't be available to you by default. You will need to set the disableRetroCompatibility property to true in the licenseStorage option (itself part of the keySystems `loadVi...

v3.23.1

Release v3.23.1 (2021-02-01)

This very small release only fixes a single issue, brought with the v3.23.0 release published this same day, which made it impossible to play encrypted contents on Safari.

Sorry about that, we normally test that specific use case, but we didn't after a quick code update* we did just before releasing the v3.23.0.

We will try to be more careful from now on and always test that use case before new releases, no matter how small the update!

* For the more curious readers, the incriminating update we're speaking of here was about throwing better errors when playing encrypted contents on platforms which do not support decryption : ed282ee. While writing that, I didn't realize that the else condition I was replacing was for a much more general case than what I replaced it with.

Changelog

Bug fixes

• Fix support of encrypted contents on Safari (v3.23.0 regression)