v3.23.0

Release v3.23.0 (2021-02-01)

This release adds multiple new methods and options to the RxPlayer, but also many other improvements.

Most importantly:

• A reload method has been added, to be able to manually quickly re-load a content after something when wrong (or even when nothing went wrong at all)

• a whole new discontinuity management system has been written, which allows the player to automatically detect and step over most "holes" in a content, even when those were not declared as such in the Manifest.
  Those type of holes are most often encountered between Periods in multi-Period DASH contents but can also be encountered in poorly encoded/packaged contents.

• A onCodecSwitch loadVideo option has been added, so you can ask the player to reload automatically when switching between incompatible video codecs or audio codecs, which you might want to do on some devices/browsers.

• multiple methods and options have been added to indicate a minimum audio and/or video bitrate to which the player should switch to when adaptive streaming is enabled

• Fallbacking to another Representation due to an in-exploitable decryption key (only when the keySystems[].fallbackOn loadVideo option is set) could be non-functional on some devices since the v3.21.1. This is now fixed.

A new reload method


We’ve noticed that there are some cases where an application wants to load again the last loaded content.
Some of these reasons could be:

• A fatal error occurred due to exceptional reasons, and thus may not reproduce after reloading the content.
• A decoding error occurred at a given playback time (e.g. the corresponding media chunk is defective), and provoked a crash. Here we can re-load while "skipping" the faulty segment.

For now, if an application developer wants to reload the exact same content, even with the same options than before, he/she needs to call the loadVideo API again. Then, the loading process will be decomposed into several steps, some of which are time-consuming: loading the manifest, communicating with the media browser APIs, etc.

This implies that after re-loading a content, all these steps will be taken for a second time. In some cases this is completely unnecessary, as fetched or parsed resources may not change from the first loadVideo to the next.

For now, we have identified one major step that can be avoided: fetching the manifest and turn it into our internal manifest format.
This operation can take a certain amount of time, which could be removed here by using the previously loaded manifest.

Thus, we’ve introduced a new API, the reload method.

It can be called at any moment, after the LOADED state for the last loadVideo has been received, even if the content is still playing.
The aim of the API is to propose to reload the last loaded content as fast as possible, by default at the last playback position it was set to.

The API options allow setting an absolute or relative (to the last playback position) position to reload playback:

// Reload the content at the last position it was set to
rxPlayer.reload();

// Reload video at a specific position
rxPlayer.reload({ reloadAt: { position: 55 } });

// Reload 5 seconds AFTER the last position it was set to
rxPlayer.reload({ reloadAt: { relative: +5 } });

// Reload 5 seconds BEFORE the last position it was set to
rxPlayer.reload({ reloadAt: { relative: -5 } });

The reload method is documented here.

Improved discontinuity management

This release contains a completely revamped discontinuity management logic.

A "discontinuity" is the name we give to holes in a media content.
For example, we could have a live content with an imperfect transition between ads and the show coming just after, where a few seconds of video would be missing.
Here the player has to do something - most likely step over that hole - to avoid getting stuck on it.

Most discontinuities were already handled in previous versions, but there were still issues we had - most often in contents concatenating multiple sub-contents (multi-Period DASH contents, "ad-switched" contents, MetaPlaylists...).

Previous discontinuity management logic

In previous versions of the RxPlayer, two types of discontinuities where handled, through a very simple logic.

When the player was stuck, it basically asked itself two questions:

• is there a very small (less than 1 second) hole at the current position? -> If yes, seek over it.
• does the Manifest tells us that there's no segment there? -> If yes, seek at the start of the next segment.

Those worked with most type of discontinuities, which are often either very small or clearly announced in the Manifest.
Yet it still had important drawbacks:

1. It was possible to have false positives: we could be stepping over what we thought to be small holes but which were just yet-to-be-downloaded segments

2. For discontinuities not announced in the Manifest specifically, the size of the "hole" had to be known before being able to seek over it.
   This meant we had first to load and push the first segment coming after that discontinuity.

   This could lead to a deadlock when loading the next segment is not yet possible.

3. Discontinuities defined in the Manifest might not reflect the exact reality once the segments are pushed: discontinuities might actually be bigger or smaller than initially announced

Solution: changing the whole logic

So we completely scrapped what we did before and thought about what the optimal way of handling discontinuities should be.

The answer we came with was the following:

1. The RxPlayer can reliably know which "holes" in the audio and/or video buffers won't ever be filled by any segment. When such a situation is encountered, it is signaled internally.

2. When one of this hole is finally encountered, playback may be stuck.
   When that happens, the player will cross-reference the position at which it is stuck with the currently signaled "holes". When a hole corresponds to that position, it automatically seeks over it.

This should completely eliminate the risk of false positives and handle almost all discontinuities whether they were clearly defined in the Manifest or not.

That new logic also comes with a smarter handling of boundaries between multiple Periods, which is where most discontinuities are encountered.

A new DISCONTINUITY_ENCOUNTERED event

Because an application might want to be be notified when discontinuities are seeked over, we also added a new warning event, emitted when this happens.
Emitted with that warning event, will be a MEDIA_ERROR instance with the "DISCONTINUITY_ENCOUNTERED" code.

You can thus know when a discontinuity has been seeked over by writing:

rxPlayer.addEventListener("warning", function (warning) {
  if (warning.code === "DISCONTINUITY_ENCOUNTERED") {
    console.log("A discontinuity has been encountered:", warning);
  }
});

Codec switching behavior

It is sometimes possible to encounter multiple different audio and/or video codecs in the same content.

For example, you might have a high-quality HEVC video track and a lower-quality AVC video track. You might also have a multi-Period DASH contents, where some Periods contain a Dolby Digital Plus audio track but others do not.

As a consequence, the RxPlayer could need to transition between two completely different codecs.
For example, the application could chose to select an HEVC video track while it was playing an AVC one, or a codec switch could just occur while entering a new Period.

The previous strategy when doing that was to just concatenate both type of segments on the same buffer while using the changeType API if available.
We found that strategy to be unreliable: depending on the device and browser, switching the codec that way could either work or totally fail. We have multiple examples of problem we encountered when doing just that on some devices: decoding error, error while appending and infinite rebuffering issues.

Another strategy would be to just reload the content any time an audio or video codec switch is encountered.
This should work for all platforms and browsers, but it means that for a small period of time, the player will go through the RELOADING state, during which the screen becomes black and most APIs are not usable. This leads to a much less pleasant experience.

From those elements, we thought that the best possible solution was to choose the first strategy (just continue pushing) when it worked and use the second one (reload) when it didn't.
We first planned to automatically detect the right strategy to take in the player (e.g. based on the presence or not of the changeType API) but we found out that doing this is very difficult and may not even be possible.

In the end, the solution we propose in this release is to let the application select the correct strategy through a new option.
An application should have a better idea of supported platforms and browsers, and the behavior when switching codecs can generally easily be checked.

This takes the form of a new loadVideo option, called onCodecSwitch, which is documented here.
It can take two values: either "continue" which is the default non-disruptive strategy (which may not work for all targets) we already have, or "reload", the disruptive strategy known to work for all targets:

// Reload every time a new codec is enc...

v3.22.0

Release v3.22.0 (2020-11-17)

Overview

This release adds multiple features and improvements:

• add an audioTrackSwitchingMode property to loadVideo to select the RxPlayer's behavior when switching the audio track (to choose between smoothness or instantaneity)
• add the initialManifest loadVideo option to avoid loading the Manifest when not necessary
• add an enableFastSwitching loadVideo option to enable or disable optimizations doing segment replacement in the browser's buffer
• the TextTrackRenderer tool is not experimental anymore. It can be used without fearing that its API will change
• A new version of the experimental local Manifest which allows much more advanced behavior, such as playing a content which is still downloading.
• A fix for an issue which made it impossible to play encrypted contents when in directfile mode
• saner defaults for TTML subtitles (white, centered text)

audioTrackSwitchingMode: behavior when switching between audio tracks

Until now when switching to a different audio track, the RxPlayer tried to make the transition as smooth as possible.

It doesn't interrupt playback of the old track, but load in parallel the new track and once done, replace the old by the new - most of the time without needing to pause playback.
Because this means that the user would still play in the old track for some time, a time limit (now of 2.5 seconds) is set from the point when the user chose a different track. If he/she has played until that limit, the player will enter a BUFFERING phase until the new track is available.

This behavior allows a very smooth experience but has two drawbacks:

1. It means that the switch won't happen right away.
   For example when switching between languages, you might prefer to interrupt the track in the previous language to play the right one at the same position you were in when you switched, even if it means interrupting playback for a while.

2. On some peculiar devices, multiple issues could arise when replacing those tracks smoothly. Sometimes it is not taken into account, sometime it doesn't even work.
   Note that we're not speaking of the usual web targets here but only of a minority of embedded devices.

Thus, we now let the possibility to adopt different strategies depending on how the audioTrackSwitchingMode API is set:

• Either, "seamless", which is the default behavior, where the transition will appear smooth and will not interrupt playback but could take time to be effective.
• Or, "direct", it will be much more instantaneous but it is possible that the player will go in "RELOADING" state in the meantime and so, may interrupt playback during a brief time.
  Note that we try to avoid going into the "RELOADING" state. We only do it when it is really necessary.

This property is a loadVideo option documented here.

For example to go into a more "direct" track switching, you can do:

rxPlayer.loadVideo({
  // ...
  audioTrackSwitchingMode: "direct",
});

initialManifest: provide the Manifest file if already downloaded

At Canal+, we encountered situations where the application calling the RxPlayer needed to process data from the Manifest before loading a video.

As such, the Manifest would be loaded twice. Once by the application beforehand and then by the RxPlayer.
This mean that we would perform an unnecessary request which would be both a waste of time and resources.

To load faster when the Manifest is already available, we consequently just added the initialManifest option to the transportOptions of loadVideo:

rxPlayer.loadVideo({
  // ...
  transportOptions: {
    initialManifest: myManifest,
    // ...
  }
});

As using it has some implications for live contents, we advise to first read the corresponding API documentation, in the transportOptions before using it.

enableFastSwitching: avoid segment replacement on some targets

The RxPlayer always try to play the best possible quality by default.

For example, when you begin to play a content at a low quality (because of poor bandwidth or because this is the initial content) but the bandwidth indicates that we can now download segments of a much higher quality, the player will try to directly replace the low-quality segments it has in its buffer by the higher quality segments it can download.

This lead to a much faster visible change in quality than if the player only loaded and pushed segments coming after those already loaded.

This optimization - replacing low-quality segments with higher quality-segments for raising faster in quality - has been integrated in the player for multiple years, but was given a name only recently: "fast-switching" (name admittedly taken from another player, dash.js as they developped a similar optimization).

By default, "fast-switching" is enabled and has been for a long time. But it's now possible to disable it through a new loadVideo option: enableFastSwitching.
By setting it to false, the player will just push new segments at the end of the buffer, regardless of the quality they are in.

The main reason you would want to disable that feature, and the main reason behind making that option configurable, is that some specific environments react very poorly when segments already in the buffer are replaced.
For example, we noted that some rare environments do not consider the new segments (making that optimization useless). We also worked recently on an environment which reacted very poorly to this optimization, leading sometimes to the video "freezing".
This was for a very specific environment which was still in heavy development and they since fixed that issue, but it was the main trigger behind making this behavior configurable, as anyone could encounter the same situation on a new device.

So you should probably not put it to false (even if there's no problem in doing so beside slower change in quality) but it can be useful when you know what you're doing.
You can consider this as an "expert" option!

This option is documented here.
It is set to true by default. To disable fast-switching, you can thus do:

rxPlayer.loadVideo({
  // ...
  enableFastSwitching: false,
})

The TextTrackRenderer is not experimental anymore

The TextTrackRenderer tool, which was added in the v3.18.0 version of the RxPlayer, allows to show any subtitles in the supported formats on top of a media element (it both parses and synchronizes them), even when the content is not played through the RxPlayer.

Until now, the API was marked as "experimental" which meant that its API could change at any time.

It now has a stable API, meaning it's not experimental anymore!

Note that if you already used it, you now have an action on your side to be compatible to the v3.22.0.
When before you imported the TextTrackRenderer as such:

import TextTrackRenderer, {
  TTML_PARSER,
  VTT_PARSER,
  SRT_PARSER,
  SAMI_PARSER,
} from "rx-player/experimental/tools/TextTrackRenderer";

You now have to get rid of the /experimental tp instead write:

import TextTrackRenderer, {
  TTML_PARSER,
  VTT_PARSER,
  SRT_PARSER,
  SAMI_PARSER,
} from "rx-player/tools/TextTrackRenderer";

As usual, its API documentation is available here.

Default TTML style

We encountered an issue where a user of the RxPlayer library basically told us that we badly formatted TTML subtitles, he couldn't read them well as they were in a black color and they were displayed on the top left of the screen.

After some investigations, we found out that we had no big problem with TTML formatting but that the real issue is that we do not make styling decisions if no style is present in the TTML file itself (and generally, when you are using TTML subtitles, the style is included!).
This means that by default, only the already-present styles of the concerned HTML elements are considered.
On most cases, this means black and on the top left of the screen!

Resolving that issue was not straightforward, because no default style seems to be defined in the TTML specification.
So we weren't sure of what to do here.

After looking at what other players were doing and at what the most sensible thing to do in that situation would be, we now consider that if no style at all is declared in a TTML file, we should apply a default style at its place.
Therefore, if no styles are defined for regions in the TTML, but other styles are (e.g. on HTML elements), we decide not to apply the default style. Indeed, we consider the style to have been defined on purpose, and we may interfere with the overall look of the cue if we apply a different style.

That default style objective is to display text cues in the clearer and cleaner possible way: at the bottom, centered and with white text

serverCertificate optimizations

The serverCertificate property of the keySystems option (in loadVideo) allows to communicate a server certificate.
This is DRM-specific data usually needed to encrypt message between a Content Decryption Module (or CDM) and a license server.

It appears that setting that certificate can take some time: up to 1 second on some embedded devices.
We thus decided to allow a new optimization: when we know a serverCertificate is alr...

v3.21.1

Release v3.21.1 (2020-09-21)

Overview

The v3.21.1 adds many small bug fixes and improvements:

• we fixed an issue that made it difficult to switch between key systems (e.g. from Widevine to PlayReady) on devices and browsers that support it
• we improved the support of DASH MPDs, in particular we now support contents with multiple levels of SegmentTemplate declaration
• we fixed some issues with the manifestUpdateUrl API, which allows to optimize the performance when playing very long Manifest files.
• we brought several small improvements to reduce the delay at which a track is switched and seek are performed and to improve the experience when switching quality
• we updated our adaptive logic so it favours even more smooth playback over maximizing the quality
• many other things, see our changelog on the bottom of this release note for more information (now with links on related PRs and issues!)

Switching between key systems

To be able to play encrypted contents on the browser, a CDM - or Content Decryption Module is used.

That module is usually an external software, plugged to the current browser, whose task will be to do what's needed to decrypt the content.
They are provided by DRM technologies whose names you have probably heard of if you play encrypted contents: Widevine, PlayReady, FairPlay etc.

 How encrypted contents are decrypted via the RxPlayer (simplification):

     +===================+
     |    Application    | Provide wanted DRM configuration(s)
     +===================+ and the logic to fetch a licence
              |
              V
        +----------+
        | RxPlayer | Try to open a session with a CDM having the
        +----------+ wanted configuration.
              |      Once opened, fetch and give the licence - allowing
              |      to play the content - to the CDM.
              V
  +-------------------------+
  |       Web browser       | Provide APIs to interact with CDM(s)
  +-------------------------+
              |
              V
           +-----+
           | CDM | Decrypt the content
           +-----+

Because multiple CDMs exist and because they might have different capabilities depending on the device on which the content plays (for example, devices made for video streaming might have more copy-protection mechanisms), not all browsers or devices use the same ones.

For the most part, we're stuck with only one possibility. For example most Chrome and Firefox builds on a desktop/laptop computer rely on the Widevine CDM and no other.
But other situations exist where multiple CDM can be available at the same time. For example the new Edge browser on desktop will usually allow us to interact either with PlayReady or Widevine. That exact situation is also found on other environments, like on recent Samsung TVs.

  +-------------------------+
  |       Edge Browser      |
  +-------------------------+
       /            \
      /              \
     |                |
     V                V
 +--------+      +---------+
 |  CDM   |      |   CDM   |
 |Widevine|      |PlayReady|
 +--------+      +---------+

The RxPlayer API let you choose which CDM you want by specifying what it calls a "key system": you can play a content with a Widevine CDM by claiming that you want the "widevine" key system.

All that work well as long as the corresponding key system is available. However, there was one situation where the RxPlayer was not so much reliable: when switching from one key system to another one, with the same RxPlayer instance.

Some browsers unexpectedly failed when "switching" the key system from one to another. After some initial tests, we thought that the situation was unsolvable: you were stuck with a key system once you've chosen one on some devices.

                 +-------------------------+
                 |         Browser         |
                 +-------------------------+
                      /            x
                     /              x
                    |                x
                    V                V
                +--------+      +---------+
                |  CDM   |      |   CDM   |
 CDM chosen  -->|Widevine|      |PlayReady|<-- We can't use that
 to play the    +--------+      +---------+    one anymore
 first encrypted
 content

However, we finally found out that by re-ordering some API calls, such error disappeared.
This is not the first time where we find that re-ordering DRM-related API calls unexpectedly unblock some situations on some devices. Because of this, we now follow a very particular sequence of calls that we know to work on all our tested platforms.

The result is that you can now reliably choose a different key system depending on the content played.

An in-depth explanation of the solution might rely on multiple technical terms (most of all MediaSource and MediaKeys) so I wont explain it in that release note. If you're still curious about it (and know what those terms roughly mean), you can look at the corresponding issue (#766) and PR (#744).

Reducing the delay of track switching and some seek operations

One of the main task of the RxPlayer is to schedule the right media data requests at the right time.

The basic behavior behind it is to download in parallel the next most needed audio, video and text (if subtitles are enabled) media "segments" (which are basically small decodable chunks of the whole media data).
When a segment is loaded, it is stored in a buffer so it can be decoded later (as soon as we need it). At the same time, we begin the request for the next most needed audio, video or text segment so we can maximize the bandwidth usage and provide smooth playback with the best possible quality.

 Queues of segment scheduled for download

     AUDIO QUEUE           VIDEO QUEUE          TEXT QUEUE
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 1 |   | video segment 1 |  |  text segment 1 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 2 |   | video segment 2 |  |  text segment 2 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 3 |   | video segment 3 |  |  text segment 3 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
         ...                   ...                  ...          

However if we did just that, we could quickly encounter a problem. For example, audio segments are generally much smaller in size than video segments, and as such they are loaded much more rapidly. If we loaded the next audio segment as soon as the previous one's request ended, we would be limiting the bandwidth for the more urgent video segment request happening in parallel.

                      AUDIO QUEUE           VIDEO QUEUE     
                  +-----------------+   +-----------------+
distant, not  --> | audio segment 7 |   | video segment 1 | <-- close, really
urgent segment    +-----------------+   +-----------------+     urgent segment
                           |                     |            
                           V                     V            
                  +-----------------+   +-----------------+
                  | audio segment 8 |   | video segment 2 |
                  +-----------------+   +-----------------+
                           |                     |            
                           V                     V            
                  +-----------------+   +-----------------+
                  | audio segment 9 |   | video segment 3 |
                  +-----------------+   +-----------------+  
                           |                     |            
                           V                     V            
                          ...                   ...                                     

To solve this issue while still making the best use of the user's bandwidth, the RxPlayer make a compromise.

First, it assigns to each segment a priority.
When a media segment is loaded, the RxPlayer will first check the priority of the next most needed segment for that type (e.g. "audio"). It will only start a request for that segment if/when its priority is at least on par with the priority of the most needed segments for the other types. This sounds complicated, but it allows to download urgent segments with all available bandwidth while still being able to parallelize segment requests once urgent segments are all downloaded.

    AUDIO QUEUE           VIDEO QUEUE     
                      +-----------------+
                      | video segment 1 |
                      | (high priority) |
   Waiting until      +-----------------+
  higher priority              |            
  video segments               V            
    are loaded        +-----------------+
                      | video segment 2 |
       ...            | (high priority) |
                      +-----------------+
                               |          
                               V          
+-----------------+   +-----------------+
| audio segment 7 |   | video segment 3 |
| (lower priority)|   | (lower priority)|
+-----------------+   +-----------------+
         |                     |         
         V                     V         
        ...           ...

v3.21.0

Release v3.21.0 (2020-06-17)

Overview

The v3.21.0 brings multiple features amongst which:

• the RxPlayer now handle DASH EventStream's Event elements to emit timed events defined in a DASH MPD
• it is now possible to apply track preferences to the content(s) / Period(s) that are already playing.
• WebVTT "settings" are now considered when parsing subtitles in the html textTrackMode, thanks to @mattiaspalmgren for the contribution! Before, we only relied on WebVTT's CSS extensions to allow stylization of subtitles in this mode.
• the RxPlayer will now emit "warning" events when it detects minor errors in a DASH MPD, such as optional data in the wrong format.
• and a lot of other bug fixes and improvements - you can look at the changelog for more information.

Support of DASH EventStream's Event elements

What stream events are for

What we call "stream events" here is metadata synchronized with the content being played. There are several reasons why you might want to include those.

At CANAL+ for example, we started to study the implementation of ad-insertion and tracking solutions for live contents. Ad-insertion can work on the server-side by generating an MPD which will replace media segments by ads, for example by adding Period elements.

On the client-side (the media player), we then may want to know where an ad can be encountered and when the final user begins to see it (or when he/she skip it) for tracking purposes.

Here, DASH EventStream's <Event> elements can be used.
Such elements define timed events associated with the current content. When the user begins to play at a position within the time boundaries of that event, the corresponding Event element will be considered.
This element can contain specific user-defined information, such as the URL of an ad-tracking server.

But DASH <Event> elements are not restricted to advertising use cases, you could use one anytime you want to be notified when an user is at a particular position in the stream.

How the RxPlayer handle DASH EventStream elements

DASH's <EventStream> are elements of the MPD, inside the corresponding <Period> element.
Each EventStream can carry one or more <Event> elements, whose format and inner data depends on an attribute of the EventStream element called the schemeIdUri:

    <Period id="1">
      <EventStream schemeIdUri="urn:uuid:XYZY" timescale="1000" value="call">
        <Event presentationTime="20000" duration="10000" id="1">
          Some data
        </Event>
        <Event presentationTime="40000" duration="10000" id="2">
          Some other data
        </Event>
        <Event presentationTime="60000" duration="10000" id="3" />
      </EventStream>
      <!-- ... -->
    </Period>

Each Event element generally has an assiociated "presentation time" (the initial time the event should be triggered at), some specific data and, sometimes, a duration.

The RxPlayer acts like a passthrough with the Event's data.
Its only role here is to send "streamEvent" events when a user goes into the time boundaries of such events. By catching that event, an application can also be notified when the user "exits" the same event's time boundaries:

// Do something when the user enters a time area linked to an Event element
rxPlayer.addEventListener("streamEvent", (evt) => {
  console.log("Beginning of an event:", evt);

  // Catch the moment when we are exiting the event.
  // Note that only Event elements with a duration (and thus an end) can
  // have an "exiting" notion.
  if (evt.end !== undefined) {
    evt.onExit = () => {
      console.log("End of the event:", evt);
    };
  }
});

The user could also "skip" an event, like when seeking after it. As an application, you may want to know when that happens.

For this reason, the RxPlayer also reports when an event has been "skipped" through a new "streamEventSkip" event:

rxPlayer.addEventListener("streamEventSkip", (evt) => {
   console.log("An event has been skipped.", evt);
});

Other types of events

For now, only DASH EventStream are handled by the RxPlayer but several types of events exist.
DASH also has in-band events that are in the media containers themselves like for example in "emsg" ISOBMFF boxes.
Other streaming protocol have their own event format(s) as well.

We want to support more of them in future versions, the available APIs here were also written with those in mind.
The management of "emsg" boxes, for example, is something we are today considering to add in our next versions.

More information on those APIs

Because the concept and APIs could seem complex we have added both those APIs to our API documentation and added a new tutorial about it.

We advise people wanting to integrate stream event management to first read the tutorial and to then come back to the API documentation for checking what are the arguments awaited.

The tutorial is here.

And here is the documentation of the new APIs:

• the streamEvent event for when an event is encountered
• the streamEventSkip event for when an event is "skipped".

We also export two new types (which can be imported via "rx-player/types"):

• IStreamEvent: the object sent with a streamEvent or streamEventSkip event
• IStreamEventData: the value of the data attribute of an IStreamEvent.

Those are documented here.

Applying track preferences to the current content

The two APIs for track selection

When it comes to track selection in the RxPlayer, we have two set of APIs:

• the "classic" track selection APIs - like getAudioTrack, setVideoTrack, getAvailableTextTracks - will tell you which tracks are available and allow you to choose a specific one from that list

• the track preferences APIs - like setPreferredAudioTrack - allow to define a general wanted choice. For example, we could indicate that the RxPlayer should choose by default a french audio track and/or english subtitles if those are available.

The APIs in the first set list the available tracks, so the application can then choose a precize one.

The APIs in the second set let the RxPlayer do the choice itself based on criterias. They are also used in some internal RxPlayer optimizations, like when pre-loading a future content - where the right track to download is for now "guessed" by the RxPlayer.

The problem with track preferences

The track preferences APIs have thus many advantages, but they have a limit: they only apply preferences for future contents and periods.

The content which is currently played and the Periods (for DASH) or sub-contents (for MetaPlaylist) that we played previously will still remain with their last set track configuration.

This is by design, we didn't want to change the current tracks after a setPreferred... call because this could seem unexpected.
So we are left in what we could refer as a "hole" in our API:

• We can define the track for the current Period (or sub-content) but this won't affect any other Periods
• We can define a preference for new Periods but this won't affect the current one or the one that have already been played.

What if we just want to set a new global preference, that we want to apply retroactively (to the current and already-played Periods)?

We could call both APIs, but we would still be left with the previous Periods in their previous track configurations.

This problem can be seen for example when you want to enable an "audio-only" mode (by disabling the video track) for multi-period contents:

// The old way

// disable the current track
rxPlayer.disableVideoTrack();

// Update the preferences for the next content / periods
rxPlayer.setPreferredVideoTracks([null]);

But here seeking back to a previous period is going to display the video track again. To work-around that, we have to call disableVideoTrack() again when seeking back to them.

A solution

We did many drafts on how we could improve our track selection APIs so that all possible use-cases are handled.

Among those, we thought of giving "more power" to the classic track selection APIs, so they could also change the track for other Periods.
However, we found that that approach would necessitate a very complex API. For features as simple as applying a global preference, we thought that the API should stay simple.

In the end, we decided to add an optional boolean as a second argument to those track preferences methods:

• setPreferredAudioTracks
• setPreferredTextTracks
• setPreferredVideoTracks

When set to true that preference will be applied retroactively.

For example, disabling the video track globally now can simply be done this way:

rxPlayer.setPreferredVideoTracks([null], true);

A new tutorial on track selection

As you can see, the RxPlayer allows applications to have a complex track selection management. However, this can come at the cost of ...

v3.20.1

Release v3.20.1 (2020-05-06)

Overview

This is a small release which mainly contains bug fixes.

It most-of-all fixes a small regression brought in our v3.20.0 release.
This issue could lead users to not be able to play multiple encrypted contents. It was though relatively rare to encounter, as it was only triggered in the following situations:

• when using Internet Explorer 11 and playing multiple encrypted contents

• when linking more than RxPlayer instance to the same media element - after calling rxPlayer.dispose() to dispose the last one - and playing encrypted contents in more than one of such instance

For applications which did not dispose and re-instanciate the RxPlayer on the same media element and did not target IE11, this issue should have had no impact. Likewise, applications not playing encrypted contents are not concerned by this issue.

We also added other fixes to this release not linked to this (or any) regression.

Regression when disposing of a MediaKeys instance

The main issue here (summarized in the overview) was due to the way we changed our DRM-related code to better support Safari.

When doing that, we updated the code attaching MediaKeys instances (the main interface involved in content decryption) to the media element to be adapted to both the regular and Safari way of doing so.

However, we poorly considered and tested the possiblity of setting a null MediaKeys, which is usually what you want to set when disposing of the current instance attached to the element.
When wanting to set a null MediaKeys, the RxPlayer just ignored it instead.

This was done in two situations:

• between each content when playing in Internet Explorer 11, to work-around an issue on this platform
• after calling the rxPlayer.dispose() method, to dispose of that resource (the MediaKeys instance)

This wrong behavior only became an issue when loading a new encrypted content on the same media element, where the same MediaKeys logic would unexpectedly throw.

Fixing this issue was just a matter of better handling MediaKeys disposal, which is now done.
We do not detect an issue on IE11 or on applications calling dispose anymore.

DRM issue for contents with multiple keys

This is a rare and subtle issue that could have impacted very sporadically some applications playing encrypted contents - with several keys per content.

As such, I'll try to be very descriptive on what the problem was and how it could have had an impact.

What's a MediaKeySession?

A MediaKeySession is the browser API allowing to communicate decryption keys to the Content Decryption Module integrated in the browser, which tasks is to decrypt the content.

We create a single MediaKeySession per initialization data received, which roughly means one per license needed to decrypt the content.
In most encountered cases where we only have one key per license, we can thus simplify this concept by saying that we could need to create as much MediaKeySession for a single content as the number of keys in that content.

The MediaKeySession cache

We try when possible to keep old MediaKeySessions in a cache even when other unrelated key(s) are needed. Doing so has several advantages, like speeding-up the loading of an already-played content, the ability to avoid license requests we already have done etc.

This cache will store a maximum of 50 MediaKeySessions at the same time.

The problem

It all becomes more complicated when we consider another cache maintained by the RxPlayer, listing which encryption information has already been encountered in the current content.

That second cache is needed for a similar (but different enough) reason than the MediaKeySession cache: to only trigger some logic when new encryption data is received, and avoid duplicating such logic in other cases, for example when the same data is received two times in a row.

Where the problem lies is that the MediaKeySession cache has a logic to evict old stored data but the second cache - let's call it the InitializationData cache - has none, and there is some dependency between them due to how the RxPlayer consider both.

To explain this, let's consider an example:
We're playing a content with 51 keys, each one associated to a different license. Each time we're encountering a media segment needing a new key - and only in that case - we would be creating a new MediaKeySession, then loading its license and so on.
When reaching data needing the 51st key, we will remove the MediaKeySession linked to the 1st key to make place for the one linked to the 51st.
If then we encounter data related to the 1st key again, the InitializationData cache would ignore it, as it has been already encountered. But here because its linked MediaKeySession is now closed, we're not able to play that data. We will be left waiting for a decryption key which will never come.

This case could be considered extremely rare because encountering that much encryption data on a single content is in itself extremely rare.
But this issue could arise in more subtle ways, such as when zapping between multiple contents and playing them again, as those 50 MediaKeySession do not need to be from the same content.

Note that this problem was only an issue when playing a content needing several licenses. In any other cases, even when switching a lot between contents, everything should have worked fine.

The solution

The behavior for both cache stays the same, we do not limit the InitializationData cache as it wouldn't make sense when considered alone.

To resolve that problem in an other way, the RxPlayer core logic now removes manually entries from the InitializationData cache when the corresponding MediaKeySession cache entries are evicted. As both caches are not handled in the same place, this is done thanks to an event.

Avoid hash collision in DRM logic

Another issue was close to the one we just described: some encryption data could be ignored because the RxPlayer wrongly thought it has already been encountered.

This is because we do not compare the data itself, we first hash it (through a custom hashing function) and then use that hash for comparison. If two different encryption data give the same hash, the RxPlayer will wrongly consider both to be the same.

This is however limited by the fact that such collision should be very rare, as far as we know this case has never been encountered. Still, the cost of comparing the encryption data itself is very low, so low that we considered this "optimization" of only checking the hash as an unnecessary risk.

Now the initialization data is still hashed, but when the same hash is encountered, we still compare the encryption data linked to that hash.
That way we still profit from the speed at which a hash can be used for lookup while still protecting against collisions with a very small cost.

Switching back to the preferred audio, video or text track in a Directfile content

There was a minor issue with directfile contents and track management where an audio, video or text track set by the user could be replaced by the most "preferred" one (according to the preferred{Audio,Text,Video}Tracks APIs).

This was because on directfile contents, we run a logic setting the optimal track everytime the list of available ones change. The definition of "optimal" there was just based on the preferred tracks, without considering that the user could already have selected a track through the set{Audio,Text,Video}Track APIs.

In the case where that track is still available, it would be a weird behavior to reset the choice because a new unrelated track has been removed or added in the process.

This is now fixed. Now when a track was manually selected through the set{Audio,Text,Video}Track APIs, we stay on that track as long as it stays available.

Changelog

Bug fixes

• eme: fix OTHER_ERROR issue arising when loading a new encrypted media when a previous since-disposed instance of the RxPlayer-played encrypted contents on the same media element
• eme: fix OTHER_ERROR issue arising on Internet Explorer 11 when playing more than one encrypted content
• eme: fix issue where more than 50 active MediaKeySessions at the same time could lead to infinite rebuffering when playing back a previous content with multiple encryption keys
• directfile: for directfile contents, don't reset to the preferred audio, text or video track when the track list changes
• eme: remove any possibility of collision in the storage of EME initialization data. The previous behavior could theorically lead some initialization data to be ignored.
• eme: fix typo which conflated an EME "internal-error" key status and an "output-restricted" one.

v3.20.0

Release v3.20.0 (2020-04-22)

Overview

The v3.20.0 brings new features and improvements:

• With the addition of the disableVideoTrack method, it's now possible to play in an "audio-only" mode

• We added a preferredVideoTracks RxPlayer option, as well as setPreferredVideoTracks and getPreferredVideoTracks methods to specify which video tracks you would prefer based on its codecs and accessibility status. It's also possible to use those APIs to begin playing without any video track active.

• We added (thanks to @mattiaspalmgren) a signInterpreted property to video track APIs which signal when a video track contain a sign language interpretation (according to the DASH-IF IOP).

• We improved the audio preferences APIs to here also be able to specify a preferred audio codec.

• We added new optimizations for supporting very large MPDs (DASH's manifest)

• The Safari-specific EME implementation (for playing contents with DRM) should now be finished. Previously, you might not have been able to play DRM-ized contents with Safari.

disableVideoTrack

The ability to play in an "audio-only" mode was something we wanted to provide for some time now.

However, removing or adding a video buffer during playback is not something well supported on web browsers. In more exact terms, all SourceBuffers had to be created before we can start to push media contents to either of them. For this reason adding the possibility to disable and re-enable the video track while playing is usually not doable.

Thankfully, we already added the ability to "reload" the content (and its associated "RELOADING" state) in the v3.6.0 (august 2018) release. By simply doing that when the user disable or re-enable the video track - and some other code changes - we were able to provide this feature (those "code changes" being actually a little more complicated than how it sounds as it's the reason why we waited so long :p).

Disabling the current video track is now possible by calling the disableVideoTrack method:

function enableAudioOnly() {
  player.disableVideoTrack();

  // If you don't want subtitles to display, which may or may
  // not be something you want, you can also do
  // player.disableTextTrack();
}

Please note that doing so might lead the RxPlayer to reload the content for a very short time. While doing so, the video element will usually be reset to black and multiple APIs won't be usable. You will know when the player is reloading or not by checking when it goes in and out of the "RELOADING" state.

You can then re-enable the video track by using the usual track switching method: getAvailableVideoTracks, getVideoTrack and setVideoTrack:

function quitAudioOnly() {
  // example by just setting the first track
  const tracks = player.getAvailableVideoTracks();
  if (tracks.length !== 0) {
    player.setVideoTrack(tracks[0].id);
  }
}

It is also possible to begin without a video track enabled through the new preferredVideoTracks option, which is described later in this release note.

On a related note, you have to consider that, just like disableTextTrack, disableVideoTrack will only disable the video track for the current Period.
If you're playing a multi-Period content and if you want to disable the video track for every Period, you could either call disableVideoTrack at each periodChange event, or you could use the new preferredVideoTracks APIs (which are documented in this same release note) before loading the content.

We added the possiblity to disable the video track in our demo page if you want a quick peek of how it can look like.

signInterpreted accessibility info

Before writing about the new video track preference APIs, we need to add a note on the new signInterpreted boolean property added on video track APIs.

This new property is set to true when the corresponding video track contains a sign language interpretation.
We can extract this information from a manifest, thanks to accessibility features available in some DASH contents (you can refer to the DASH-IF IOP for more info).

This boolean was added to the object returned by getAvailableVideoTracks and getVideoTrack:

/**
 * Returns only video tracks containing a sign language interpretation.
 */
function getAvailableSignInterpretedVideoTracks() {
  return player.getAvailableVideoTracks
    .filter(track => track.signInterpreted === true);
}

/**
 * Returns true if the current video track contains a sign language
 * interpretation.
 */
function isCurrentTrackSignInterpreted() {
  const videoTrack = player.getVideoTrack();

  // videoTrack can be `undefined` or `null` for different reasons (see doc)
  if (typeof videoTrack === "object") {
    // Note: signInterpreted can still be `undefined` - when we don't know
    return videoTrack.signInterpreted === true;
  }
  return false;
}

Big thanks to @mattiaspalmgren for implementing this feature.

preferredVideoTracks

It is now possible to set a video track preference through the following new APIs:

• the preferredVideoTracks player option, when instantiating the RxPlayer
• the setPreferredVideoTracks method, at any time after instantiation

Exactly like the related audio and text APIs, those two allow to tell the RxPlayer which video track it should initially choose, according to various constraints of your choosing.

For now, you can only set those constraints in function of:

• the wanted video codecs
• whether the track should contain a sign language interpretation or not

The syntax is well explained in those two method's documentation, but let's still get through simple examples so you can quickly grasp what they can do.

Let's say that you would prefer video tracks with sign language interpretation. To spice things up, we will also prefer if the track contains only Representation in the vp9 codec. Here is what we could do:

const player = new RxPlayer({
  // ...
  preferredVideoTracks : [
    // at best, have both sign language and only the VP9 codec
    {
      signInterpreted: true,
      codec: { test: /^vp9/, all: true /* Representations should all be vp9 */ },
    },

    // If not found or supported, just sign language is fine
    { signInterpreted: true },

    // If not available either, fallback on track without sign language, but
    // still with VP9
    { codec: { all: true, test: /^vp9/ } },

    // Note: If the last one was still not available, we will here still have a
    // video track, which do not respect any of the constraints set here.
  ],
})

You could also do it at any time after instantiation, thanks to the setPreferredVideoTracks method:

player.setPreferredVideoTracks([
  { signInterpreted: true, codec: { all: true, test: /^vp9/ } },
  { signInterpreted: true },
  { codec: { all: true, test: /^vp9/ } },
]);

Last but not least, it is also possible to indicate that you wish that no video track is active, by adding a null in this list of preference.

You could add it as the only element to that array, meaning you want to play in audio-only (without a video track):

// As usual either through the constructor
const player = new RxPlayer({
  // ...
  preferredVideoTracks: [null],
});

// or through the setPreferredVideoTracks method
player.setPreferredVideoTracks([null]);

You could also fallback to no video track if none your constraints are supported, by putting null at the end of that same array. For example, if you want the vp9 codec or no video track, you can do:

// Through the constructor
const player = new RxPlayer({
  // ...
  preferredVideoTracks: [
    { codec: { all: true, test: /^vp9/ } },
    null // if not found play without a video track
  ],
});

// or through the setPreferredVideoTracks method
player.setPreferredVideoTracks([
  { codec: { all: true, test: /^vp9/ } },
  null // if not found play without a video track
]);

You can retrieve the last set list of preferrences at any time through the new getPreferredVideoTracks method.

improved preferredAudioTracks

We also improved the audio track preferences APIs by allowing you to set constraints on the wanted codecs.

Both setPreferredAudioTracks and the preferredAudioTracks constructor option now can take a new codec property.

When set, this property allows to declare that you would prefer that an audio track contains either:

• only Representation with a given codec
• at least one Representation with a given codec

As you might only want to choose a preference based on the codec and not on the language anymore, we also made the previous properties (both language and audioDescription) optional.

Let's look at examples. Let's say that you would prefer audio tracks containing Dolby Digital Plus only (also known as "ec-3"), you could do:

player.setPreferredAudioTracks([
  {
    codec: {
      test: /ec-3/, // Check for Dolby Digital Plus
  ...

v3.19.0

Release v3.19.0 (2020-03-11)

Overview

The v3.19.0 brings several new improvements:

• the RxPlayer will now be able to consider multiple URL per segment (for DASH contents)

• we've made several improvements related to the performance of the RxPlayer's Manifest parsing logic

• we've added yet again another external tool (still experimental) called createMetaPlaylist which allows to easily concatenate multiple DASH/Smooth (or even MetaPlaylist!) contents into a single MetaPlaylist content

• the RxPlayer now exports more types (for TypeScript) related to audio/video/text track choices, with 6 new types

• we fixed several minor issues. Most being related to recently released experimental tools and features

Multiple URLs per segment

DASH's MPD can declare multiple URL per segment through the declaration of multiple BaseURL elements.

This allows for example to fallback to another CDN when the main one become unavailable. It may also sometimes be used when a first source might not have all media segments yet.

The strategy taken by the RxPlayer when encountering multiple URL linked to a segment is the following:

1. It will try to request the first URL (the one described by the first BaseURL element encountered in the MPD)
2. If it fails, it will immediately request the second URL
3. If it also fail, it will immediately request the third one, and so on
4. If all URL fails, the usual retry logic starts: depending on your configuration, the RxPlayer will wait a small delay - which will grow exponentially each time the batch of requests fails - and after that delay it will try again to request each URL, one immediately after another (it's actually a little smarter than that, only requests which have failed with an error considered as retry-able will be retried. e.g., an URL returning an HTTP 403 Forbidden will not be retried).
5. If all requests have failed and either:
   - we have done the maximum number of retry configured
   - or all URLs failed on an error that the RxPlayer judged as not retry-able (like a 403)
   Then, the player will stop on error and emit the usual PIPELINE_LOAD_ERROR error event

So basically, the usual retry logic is not done between different URL (as those will usually be linked to different CDNs). All requests are batched together and the retry logic is done when all this batch fails.

This was done as it seems the most logical way to go with such use cases.

Please open an issue if that behavior does not suit you. In that case we will see whether we could update it or at least add supplementary configuration options to adapt to it.

Manifest parsing improvements

Many changes in this release are linked to performance improvements on the RxPlayer's Manifest-parsing logic.

We tried to:

1. reduce the time taken by the RxPlayer to parse a Manifest
2. do that parsing logic less often
3. split that work in multiple parts, so we avoid blocking the main thread for too long

but why?

We were confronted to the need to improve on those points because we were having performance issues on some "embedded" devices (more specifically on devices with low memory such as low-end set-top-boxes and limited devices like the ChromeCast).

This was also linked to the fact that Canal+ Group - the company which started the RxPlayer development and which employs the main RxPlayer contributors - distributes live DASH contents with a pretty big timeshift window (usually set to 8 hours).

We think that huge Manifests like those is in fine a good thing for end users, because it means an increased possibility to seek in previously broadcasted contents - contents which may not be available elsewhere.
Moreover, we just feel that it should be part of our job to better handle large contents without issues - much like we consider it would be part of the job of a text editor to handle large enough text files while still being fully usable (not thinking about any editor in particular here, this is just a - perhaps loosy - analogy).

After this long introduction to why we did some improvements let's jump into what we did.

Doing updates through a "shorter" Manifest through manifestUpdateUrl

One of the most effective change we did to reduce Manifest parsing time (more specifically update time for live contents) was to allow the consideration of two version of the Manifest for the same content:

1. The regular full Manifest
2. A shortened version of it with a smaller timeshift window

This sadly means that, to profit from that improvement, you will surely need to update the packaging logic of your streams.
But rest assured: the content does not have to be re-encoded. Both Manifest stay linked to the same content, just the size of their timeshift window change (such as DASH's timeShiftBufferDepth attribute).

In the RxPlayer, the first version of the Manifest (the full one) will in most cases only be used to perform the initial load of the content. It also may need to be requested at other times, e.g. when the RxPlayer thinks that its internal Representation of it is completely out-of-sync, but those cases are very rare and will in most cases not happen.

For regular Manifest updates, if needed, the RxPlayer will only request the shorter version.
Because it will still have information about newly generated segments from it and because the RxPlayer can just "guess" which old segments are not available anymore, no information will be lost compared to when the RxPlayer use the full version instead - as it did until now and still do if no short version of the Manifest is provided.

This optimization has a huge effect on performance. To simplify, we could say that the more there is a size difference between the full and the short version, the more that improvement grows.

Still, it does not improve the initial load time, as it only impacts Manifest updates.

If you want to profit from that optimization, you will have to set in loadVideo the new transportOptions property called manifestUpdateUrl. Every properties from transportOptions are documented here

DASH "lazy-parsing"

The RxPlayer will now only parse some parts of a DASH MPD at the time it needs them.
For example, only Period that it plays and only AdaptationSets (tracks) that it needs will be parsed, the rest won't be until we switch to the corresponding Period or AdaptationSet.

We called this behavior "lazy-parsing" as it is inspired from the lazy evaluation concept we can find in programming languages like Haskell.

It is generally beneficial but has some setbacks:

• the RxPlayer still needs a lot of information at parsing time, meaning that in many cases the improvement will be minimal (for example, it might need to parse every SegmentTimeline element in the last Period to calculate the available "edge" of a live content)
• the code become less predictable when it comes to performance issues. We could now be in a case where a simple harmless function call can trigger that lazy-parsing and lead us to several seconds of parsing (in the worst cases)

We found that this new behavior provided the most improvements on contents with multiple Periods. The more Periods a DASH MPD contained, the better the improvement was.

Also, we saw visible improvements with both the initial loading time as well as Manifest updates. This also can have an effect on VoD contents (as long as those have multiple Periods, which is usually not the case).

As this feature is always on, there is no option linked to it. As long as you have the right version, you will profit from it.

Adaptive delay between Manifest updates

While investigating what we could do to improve our performance on embedded device. we at first looked at what other DASH video players were doing.

One interesting thing we were seeing is that the shaka-player, another open-source player, didn't have as much trouble playing Canal+ DASH streams than the RxPlayer had.
The curious thing is that, by looking at their MPD-parsing logic, we didn't see much differences to ours in terms of efficiency. The improvement had to be from elsewhere.

After several false lead on how they were doing it, we finally spotted the main reason: a reduction in the frequency of MPD updates. On the shaka-player, when parsing the MPD took too much time, the next updates could be postponed. For example, instead of refreshing our Manifest every 2 or 3 seconds (a low interval, but that's how our contents were), they were on some devices doing it every 12 seconds.

This was a pretty smart thing to do.
On some devices, the RxPlayer spent almost all its time parsing the MPD, leading to visible repercussion on content playback (which appeared jerky), the delay after which user interactions were taken into account and the stability of the device as a whole.

The RxPlayer has now integrated a similar logic, where long parsing time can raise the delay we will wait until we do the next Manifest update.

What's next?

All those improvements lead to a much better experience on low-end devices.
But we can still do better, most notably for the initial loading time which can still be long for some type of contents.

To improve in that regard, we are still doing experimentation.
For example we're looking if we can even improve the impact lazy-parsing can have and even the possibility of using webassembly on the Manifest-parsing logic.

This is however still in an experimental stage and we cannot tell with certitude that such features will be available in future versions.

createMetaplaylist tool

A new experimental tool, createMetaplaylist has bee...

v3.18.0

Release v3.18.0 (2020-01-30)

Overview

The v3.18.0 is a big release with its share of important new features:

• we improved our support of directfile contents by now making them compatible with most of our track management APIs

• we added two events: "seeking" and "seeked" which are triggered each time that a seek began and finished respectively, even when the seek was done to an already buffered part.

• we added a new possible transport, called "local". It allows to play downloaded DASH, Smooth or MetaPlaylist contents.

• we added a new external tool, the TextTrackRenderer, which allows to parse and display subtitles on top of a video element.

• we added another external tool, parseBifThumbnails, which parses BIF files (thumbnails containers)

• we improved our TTML subtitles support (complex style inter-inheritance, correct handling of rgba colors)

• we should now be easier to import from node (if you want to include this library in your server-side-rendering strategy)

To prepare a potential "v4", we also deprecated some APIs (see corresponding chapter below).

You might have guessed it, this is going to be a very long release note :-o

Deprecated APIs

We'll begin with the sad part. We have deprecated a small number of APIs, which means that they will remain supported until their deletion which will coincide with a future major release - probably the v4.0.0 (no date for that one for now, but we begin to think about it).

The following loadVideo options have been deprecated:

• supplementaryTextTracks, which can be replaced by using the new TextTrackRenderer tool (described below)
• supplementaryImageTracks, here you can use the new parseBifThumbnails tool instead.

Speaking of image APIs, the getImageTrackData method has also been deprecated as well as the imageTrackUpdate event. For now, thumbnails management has entirely been moved out of the RxPlayer. Most of those can still be replaced by doing a regular HTTP request and a call to the new parseBifThumbnails function.

Lastly, we deprecated the hideNativeSubtitles loadVideo option, due to it being close to useless since we allow for the "html" textTrackMode also as a loadVideo option.

Every information on why those were deprecated and how to replace them is available in the corresponding documentation page.

Improvement of directfile API support

Before this release, multiple RxPlayer properties, methods or events had no effect when playing a directfile content:

• getAvailable{Video,Audio,Text}Tracks always returned an empty array
• get{Video,Audio,Text}Track returned undefined
• set{Video,Audio,Text}Track had no effect
• the track change events available{Video,Audio,Text}TracksChange and {video,audio,text}TrackChange were never emitted
• setPreferred{Audio,Text}Tracks and the corresponding constructor options had no effect
• getPreferred{Audio,Text}Tracks returned an empty array

This is because [audio, video or text] track management when playing a content in a streaming protocol (like DASH or Smooth) is radically different than when playing directly a single file:

• In a streaming protocol, it is mostly a task performed by the RxPlayer
• In directfile mode, the RxPlayer has to rely on native browser APIs (through these native API, you can get and set native tracks, as well as receive events on track removal/addition/change).

The RxPlayer was mostly used with streaming protocols. As such, we didn't take the time yet to plug those native APIs to the RxPlayer's methods and events.
However, we realized that it was becoming more and more used in an hybrid mode, playing both type of contents (HLS being supported in directfile mode on Apple browsers is one of the reasons why).

That's why we've now connected the RxPlayer track APIs and events to these native track API.

From now, only on browsers that offer the right native APIs, you should be able to load directfile contents with several tracks and manage them. The player will now also take the preferred tracks settings into account.

Please note that if those native APIs are not present in the current browser, those features will still have no effect.
Sadly, we found that chrome and firefox (if the right flags were not enabled) only provided text tracks APIs, which means that switching audio or video tracks on those might still not work.

But to end on a good note, you should now be able to play HLS contents on Safari and benefit from RxPlayer track management.

seeking and seeked events

We added two new events "seeking" and "seeked", which are triggered respectively when the player begin a "seek" operation (it moves to another position) and when the player finished that operation.

Before those events, an application had to listen to the playerStateChange events to check when the state went in and out of the SEEKING state for the same kind of indication.
However, the player might not switch to the SEEKING state if a seek is done in an already-buffered part of the content. This is because, in those cases, the seek could be instantaneous (and as such, the player would stay in its PLAYING or PAUSED state).

local manifests: Playing downloaded contents

We've worked for some time on the ability to download DASH, Smooth or Metaplaylist contents and play them afterward.
This could be useful for example to be able to play those contents when offline.

We splitted that feature into two tasks:

1. a tool, independent to the RxPlayer, to download and store contents
2. the possiblity to play those downloaded contents through the RxPlayer

In this release we bring the second part, which means that you can now play contents stored locally (or anywhere, really), as long as you create the right custom manifest format, called the "local manifest".

More information on this feature can be found in the corresponding documentation.

Please note that we're still working on a default implementation of a content downloader to work with it.
It will take the form of a tool, and will rely by default on IndexedDB, a native browser API allowing to store large amount of data in the browser. If you want to take a look at what it will look like, you can follow the corresponding PR: #548.

TextTrackRenderer: Displaying synchronized subtitles on top of any content

The TextTrackRenderer is a new tool which allows to parse subtitles in various formats (srt, TTML, webVTT or SAMI) and display them on top of a video element (even when the content is not played through the RxPlayer).

This tool is completely independent of the RxPlayer and can be used alone. You can look at its documentation here.

You can use this tool as a replacement of the supplementaryTextTracks loadVideo option

parseBifThumbnails: parse BIF files (thumbnails containers)

Another tool, parseBifThumbnails, has also been added in this release.

This tool is a simple function which can parse BIF files.
BIF files are simple containers for video thumbnails, it is usually used to provide an indication of where the user will seek when he/she/it hovers the progress bar.

This same format is used at Canal+ as well as Netflix, Amazon Prime or Disney+ as far as we know.

The parseBifThumbnails is totally independent of the RxPlayer and can be imported without it. Its documentation has been added here.

This tool can replace most deprecated image APIs

Importing us from node

Before this release, there was an issue when importing the library in node, a strategy sometimes used on applications with server-side rendering (SSR).

This is because of two elements:

1. The RxPlayer depend on the window object in the global scope to initialize some browser-specific constants that it will need to use often.
   In node.js, the window object is not defined. As SSR rely on node.js to pre-calculate the first rendering step, it could break if the RxPlayer was in some way imported here.

2. Many exported files are written with ES6-styles imports and exports. This is usually not a problem as most module bundler process those without any issue but it became one when node.js was directly used without a bundling step before. Node.js usually rely on another style for imports and exports, the "commonJS" style.

We've decided to work around those issues, thus improving our compatibility with SSR.
From now, the RxPlayer finds out if it is in a node environnment, by detecting the lack of window in the global scope. In that case, none of the browser compatibility strategies will be applied and the RxPlayer will stay in a mode with reduced features.

For the second problem, we could theoretically fix it in several ways: generate CommonJS imports and exports instead or signal that we were "JavaScript modules" in some ways seem to be possible solutions.
However, as we didn't have time to check on those possibilities for now. we advise people in SSR scenarios to use the esm npm module for now, as a quick and easy solution.

Changelog

Features

• directfile: support most audio tracks API when playing a directfile content
• directfile: support most text tracks API when playing a directfile content
• directfile: support most video tracks API when playing a directfile content
• api: add seeking and seeked events which anounce the beginning and end of a seek, e...

v3.17.1

Release v3.17.1 (2019-12-20)

Overview

The v3.17.1 is a small release containing minor bug fixes:

• we fixed an issue which affects mostly live DASH multi-period contents, when the MPD is refreshed

• we fixed a subtitles issue which can arise when manual garbage collection is enabled through the maxBufferBehind or maxBufferAhead options

• we fixed issues with Directfile playback on iPhones, iPads and iPods.

• lastly, we added some demo improvements

Fixed issue with dynamic multi-Period MPD

DASH' Manifest, the MPD, has a concept called "Period" which basically allows to divide a content into multiple sub-contents, each having its own tracks and available qualities.

When playing a dynamic DASH content - typically a live stream - the RxPlayer might have to refresh the MPD to know about the new available segments.
When that happens - and when the MPD has multiple Periods - the player has many tasks to do behind the scene:

• for Periods that it already knew about, it has to extract the new information obtained
• for Periods that weren't known of previously (e.g. a new program), it has to add it to its internal representation of the MPD
• for Periods that aren't here anymore (e.g. a program which is now too old, or which has been removed), it has to remove it from that same internal representation

Because the RxPlayer accounts for very rare cases, like Period inserted in-between previously known Periods, the logic behind MPD updates is easy to get wrong.

And that's exactly what happened! We detected a bug when some old Periods were removed which led toward the RxPlayer not updating later Periods, ultimately leading to an infinite rebuffering stream. The user could get out of this situation by seeking far away of the current position, but that was not an acceptable resolution of the issue!

Bugs in that area is not something that is new, and we already had a very similar issue resolved in the v3.16.1. We fixed that new one and added more unit tests on top of it for previously untested cases. We hope that this is the end of such bugs!

Less aggressive subtitles garbage collection

As the RxPlayer is present on memory-constrained devices, we had to include memory-related options and methods in our API.

More specifically options linked to media garbage collection. What we mean by that is the automatic removal of media segments which are either already played or considered too far ahead from the current position.
This is generally the job of the web browser, but we still found that logic insufficient on some devices.

This is the reason why we added the setMaxBufferAhead and setMaxBufferBehind methods, which set the amount of time in seconds the RxPlayer should keep in its buffer (removing everything else), in our v2.3.0 (in mid-2017!).

When looked upon in the API, those methods just announce that they are "removing buffer". What we really mean here is that the RxPlayer will be removing ALL type of media from its buffer: audio, video and subtitles.

Audio and video is for the most part handled by the browsers, the RxPlayer here just has to call a remove method with the right arguments and that's all.

For subtitles however, the situation is more complex:

• Either we're on the default native textTrackMode where text tracks are handled by the browser, in which case there were no problem
• Either we're on the more powerful html textTrackMode where text tracks is entirely handled by the RxPlayer.

When in that second situation, we had an issue if garbage collection was enabled.

Depending on multiple factors, such as the amount garbage collected, the size of the segments containing subtitles (basically the more garbage collection was aggressive and the longer the segments last the more "chance" you had to see the bug) and the actions of the users, the RxPlayer could decide to not replace garbage collected subtitles once they are needed again.

We fixed that issue in two ways:

1. We're less aggressive when garbage collecting subtitles.
   Before, we removed all text cues as soon as it overlapped with the interval we had to clean, even when the amount of time it had to be displayed continued out of that interval. This could lead to the removal of subtitles that were in fact still useful. Now, we only remove those cues once they are entirely contained in that "cleanable interval".

2. We now handle some forgotten edge cases when re-adding subtitles on top of garbage collected one.
   Basically, we could have to re-download a partially or totally garbage collected segment if we need it again. This action of adding again a segment on top of a partial one lead to a complex logic where the RxPlayer has to make a choice between either combine or replace both segments (there even can be more than two segments involved!).
   We mis-handled some cases, which meant that re-adding segments would in some cases not actually really add the corresponding cues to the buffer.

Directfile fixes on iOS, iPad and iPod

The directfile transport in the RxPlayer API allows to play contents which are natively understood by the web browser (like webm, mp4 but also HLS on apple's browsers) while still using the RxPlayer's API.

On such contents, the RxPlayer rely on specific browser behaviors and properties. Among them, we expect the browser to "preload" a media content even before the user has asked for it to "play".

Sadly, we found that this is not always the case on browsers that we can found on Apple mobile devices (iPhone, iPad and iPod).
This means that on those browsers, the RxPlayer could stay in a LOADING state indefinitely, until the play method is called. As this is not intuitive, we now switch to the LOADED state as soon as metadata (such as the media duration) have been obtained on those devices .
Please note that this means that a BUFFERING state (or even SEEKING, due to the initial seek) can be received just afterwards, as LOADED does not mean that data is available anymore here.

Possibility to set a custom key system in the demo

In the demo, we previously restricted the choice of key systems to either PlayReady, Widevine or Clearkey.

As demo users could need to specify more specific key systems, we now also allow them to specify one by hand (please note that the entire key system name - in reverse domain name form - is expected).

Changelog

Bug fixes

• dash/metaplaylist: fix infinite rebuffering issue when refreshing multi-Period contents under specific conditions
• buffer: be less aggressive when garbage collecting subtitles (if the maxBufferAhead/maxBufferBehind options are set) to avoid useful subtitles being removed
• directfile/compat: for directfile contents, trigger directly the LOADED state on iOS/iPad/iPod browsers as those are not preloaded there

Other improvements

• demo: display clickable "play" button on the video element when autoplay is blocked due to browser policies - to help users unlock the situation
• demo: add "Other" key system to allow specifying a custom key system in the demo page

v3.17.0

Release v3.17.0 (2019-12-09)

Overview

The v3.17.0 adds a lot of features:

• We added multiple APIs related to DRMs, especially to improve the experience when playing contents with multiple keys and/or multiple licenses
• We added a new transportOptions, checkMediaSegmentIntegrity, which allows the RxPlayer to retry a segment request if that segment appears corrupted
• Another new transportOptions, minimumManifestUpdateInterval, allows to limit the frequency of Manifest refresh (which can be useful on low-end devices)
• We improved TTML support for proportional sizes (e.g. font-size, padding, origin etc.)
• a new dub boolean, returned by APIs related to audio tracks, allow to announce if the corresponding track is a dubbing audio track.
• we better handle sudden fall in bandwidth when using a segmentLoader through the corresponding transportOptions, by allowing this custom function to report the progress of a request while it is still pending, through a new progress argument.
• and multiple minor bug fixes...

Improvements for contents with multiple keys and/or multiple licenses

Some contents may be linked to several decryption keys which may be contained in one or more licenses. For example, 240p video segments may need less copy protection guarantees than 4k video segment does, thus both can have two separate keys and/or licenses.

The RxPlayer already handled contents with multiple keys and multiple licenses without problems, but stopped on error as soon as a license or key was refused.

The more sensible strategy would be to fallback from contents with a refused key or license to a decipherable content instead.
To take our previous example, devices with limited copy protection guarantees would not be able to play 4k content, but they could automatically fallback to 240p if this quality is decipherable.

That's what we worked on with this new release.
Unfortunately, a failure to obtain decryption keys can arise at different steps, depending on technology choices and the DRM strategy:

• before playing the content
• announced by the license server, which refuses to deliver a license
• announced by the CDM/Browser, by stating that a decryption key is unusable

Also, depending on your strategy, you might even want to just fail as soon as keys or licenses are refused.

That's the reason why we chose to let you decide with a complete API, through the usual keySystems loadVideo option.

For example, to automatically fallback when a key is both refused due to an internal CDM errors or due to output restrictions, you can do:

rxPlayer.loadVideo({
  // ...
  keySystems: [{
    // ...
    fallbackOn: {
      keyInternalError: true,
      keyOutputRestricted: true,
    }
});

But that's not all, you can also for example fallback even before any key is obtained, for example when the license server refuse to deliver you a license, through the fallbackOnLastTry option.

As usual, all those new APIs have been added to the loadVideo API documentation. To help you implement the code especially linked to content with multiple keys we added a new DRM tutorial which contains a whole chapter on that subject.

checkMediaSegmentIntegrity

This release adds two new transportOptions, among which checkMediaSegmentIntegrity.

This new option allows the RxPlayer to check if a downloaded segment appears corrupted after doing a request. This had to be added because we observed issues with segment corruptions on multiple occasions (both due to browser bugs on embedded devices or to delivery issues).

For example, there can be some issues with live content delivery, as the live nature of the encoding implies that content may present some unexpected defects. In the RxPlayer, we check that expected data is in line with the loaded content. (e.g. the data length announced in ISOBMFF container compared to real data length).

With this option enabled, the RxPlayer will retry a segment request as soon as it detects an issue with it, with the same retry rules than the usual HTTP 404 error.

You can opt-in for this check like this:

rxPlayer.loadVideo({
  // ...
  transportOptions: {
    checkMediaSegmentIntegrity: true,
  }
});

This option was actually already present (but hidden) in the precedent release. We only communicated it to specific people at Canal+ to verify if this new option could reduce issues such as BUFFER_APPEND_ERROR errors at a large enough scale before definitively adding it to our API.
Because we observed a consequent fall in BUFFER_APPEND_ERROR after those tests without impacting other errors, we decided to go on and officially add this option in the v3.17.0.

For the moment, the RxPlayer is only able to check ISOBMFF segments (other types of segments are always accepted for now). More informations on this option can be found on the transportOptions documentation.

minimumManifestUpdateInterval

The second transportOptions added in this release is minimumManifestUpdateInterval.

This option allows to reduce the pace at which Manifest refresh are done.
This is usually not needed for most devices.

We added that option because we observed that on some low-end devices, a high Manifest refresh pace (i.e. every 2 seconds) could make the video stutter enough for it to become unplayable.
This was probably due to the fact that the decompression and parsing done by the browser of 8 hours+ multi-lingual Manifests with short segments took a lot of memory and that some devices can't afford that much.

Allowing a greater interval between Manifest refresh did improve drastically the experience on those devices, even if this comes with its own caveats (like being aware of new segments or of other Manifest change later).
That's why we usually advise people to rely on other tactics if that becomes a problem, such as either using a numbering-scheme when using DASH or making a better usage of the r attribute on SegmentTimeline segment indexes.

This is however not always doable. If you understand the risks and are able to mitigate them (like by playing farther from the live edge), you can check this new option in the transportOptions documentation. Here is an example:

rxPlayer.loadVideo({
  // ...
  transportOptions: {
    // keep at minimum 5 seconds between manifest updates
    minimumManifestUpdateInterval: 5,
  }
});

dub boolean for audio tracks

The method getAvailableAudioTracks and getAudioTrack, the availableAudioTracksChange event and the second argument given to the representationFilter transportOptions can now return objects with a new boolean property called dub (or isDub for the representationFilter API).

If set to true, it indicates that the linked audio track is a dubbing audio track (and not the original one). We can infer this information if specified in the DASH's MPD.

The documentation of all those APIs has been updated to include this new property.

Note about the length between releases

The previous release was done a little more than two months ago which was a little unusual for us: we usually wait no more than roughly a month between releases.

The fact that we waited that long before releasing the v3.17.0 was due to some good news: we didn't encounter neither a major bug which might need a fix right away neither a major feature which would need to be released as soon as possible.

But waiting that long also means larger releases, with much longer testing periods. We had to remove several features during our review and testing period, as we didn't feel those were ready enough.
All this means that both the list of features included in this version and its release date were much harder to predict than for previous versions.

This is why we will now try to keep roughly one month (or less if a major bug or deadline appears) between releases. Maybe we will still be a little late sometimes, but at least that will be the new target we set for ourselves.

Changelog

Features

• eme/api: add keySystems.fallbackOn property to loadVideo to allow fallbacking to other qualities when encountering various key errors
• eme/api: allow to set fallbackOnLastTry on a getLicense Error to be able to fallback on other qualities when a license request is on error
• eme/api: add NO_PLAYABLE_REPRESENTATION MediaError for when every video or audio quality cannot be played
• manifest/api: add decipherable property to a Representation object
• api: add decipherabilityUpdate event triggered when a Representation's decipherability status is updated
• languages/api: add dub boolean to audio tracks (through getAudioTrack and getAvailableAudioTracks) to tell if this is a dubbed track
• languages/ttml: with TTML subtitles, support length relative to the Computed Cell Size for tts:fontSize, tts:padding, tts:extent, tts:origin and tts:lineHeight
• transports/api: add checkMediaSegmentIntegrity transportOptions to automatically retry media segments which appear corrupted
• transports/api: add `m...