Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error updating LLHLS segments when video quality changes. #1445

Open
windbella opened this issue Nov 15, 2023 · 2 comments
Open

Error updating LLHLS segments when video quality changes. #1445

windbella opened this issue Nov 15, 2023 · 2 comments

Comments

@windbella
Copy link

windbella commented Nov 15, 2023 •
edited

Please do not delete the template, by filling out the required information we can investigate your issue more quickly.

Before opening an issue see if your problem can be resolved with the troubleshooting guide.

Description

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a1",NAME="a0",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",LANGUAGE="ko",URI="https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/a0_a1/a1.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a2",NAME="a0",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",LANGUAGE="ko",URI="https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/a0_a2/a2.m3u8"
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1572000,BANDWIDTH=3144000,CODECS="avc3.640028,mp4a.40.2",RESOLUTION=1920x1080,AUDIO="a1"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v2/v2.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=822000,BANDWIDTH=1644000,CODECS="avc3.64001f,mp4a.40.2",RESOLUTION=1280x720,AUDIO="a1"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v4/v4.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=423000,BANDWIDTH=846000,CODECS="avc3.64001f,mp4a.40.2",RESOLUTION=854x480,AUDIO="a2"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v5/v5.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=235500,BANDWIDTH=471000,CODECS="avc3.64001e,mp4a.40.2",RESOLUTION=640x360,AUDIO="a2"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v7/v7.m3u8

A live streaming environment.
In the m3u8 playback situation above, suppose there are some stacked segments in a1 and a2 due to a few quality changes.
There is an issue with segment updates when stopping and restarting a1.
If the interval between stops and starts gets longer, there will be no matching segments and you won't be able to pull map data from the previous segment.

export const updateSegments = (original, update, offset) => {
  const oldSegments = original.slice();
  const newSegments = update.slice();

  offset = offset || 0;
  const result = [];

  let currentMap; // never allocated

  for (let newIndex = 0; newIndex < newSegments.length; newIndex++) {
    const oldSegment = oldSegments[newIndex + offset];
    const newSegment = newSegments[newIndex];

    if (oldSegment) {
      currentMap = oldSegment.map || currentMap;

      result.push(updateSegment(oldSegment, newSegment));
    } else {
      if (currentMap && !newSegment.map) {
        // llhls uses a skip querystring, so you need to get the map value from the previous segment.
        // but map is not allocated
        newSegment.map = currentMap;
      }

      result.push(newSegment);

    }
  }
  return result;
};

and the handleSegmentBytes function will throw an error.

Uncaught TypeError: Cannot read properties of undefined (reading 'tracks')
    at Dh (video.min.js:38:91026)
    at i (video.min.js:38:94224)
    at Oh.i.activeXhrs (video.min.js:38:94670)
    at video.min.js:38:96100
    at Pd (video.min.js:37:20244)
    at Object.callback (video.min.js:37:20950)
    at n (video.min.js:12:44288)
    at XMLHttpRequest.e (video.min.js:12:45067)
    at XMLHttpRequest.sentryWrapped (helpers.ts:85:17)

Sources

I've come up with two possible solutions.
but I'm not familiar with the structure of the http-streaming project, so bear with me.

  const updateSegments = (original, update, offset) => {
    const oldSegments = original.slice();
    const newSegments = update.slice();
    offset = offset || 0;
    const result = [];
    let currentMap;
    // added part, save the map of the last segment if there is no overlap
    if (oldSegments.length <= offset) {
      for (let oldIndex = oldSegments.length - 1; oldIndex >= 0; oldIndex--) {
        const oldSegment = oldSegments[oldIndex];
        if (oldSegment.map) {
          currentMap = oldSegment.map;
          break;
        }
      }
    }
    for (let newIndex = 0; newIndex < newSegments.length; newIndex++) {
      const oldSegment = oldSegments[newIndex + offset];
      const newSegment = newSegments[newIndex];
      if (oldSegment) {
        currentMap = oldSegment.map || currentMap;
        result.push(updateSegment(oldSegment, newSegment));
      } else {
        if (currentMap && !newSegment.map) {
          newSegment.map = currentMap;
        }
        result.push(newSegment);
      }
    }
    return result;
  };

(playlist-loader.js)

  adjustForLLHLS() {
    if (this.llhls) {
      this.started = false;
    }
  }

(add method playlist-loader.js)

export const onGroupChanged = (type, settings) => () => {
   // ...
}

export const startLoaders = (playlistLoader, mediaType) => {
  mediaType.activePlaylistLoader = playlistLoader;
  playlistLoader.adjustForLLHLS(); // added part.
  playlistLoader.load();
};

(media-groups.js)

  load(shouldDelay) {
    if (this.mediaUpdateTimeout) {
      window.clearTimeout(this.mediaUpdateTimeout);
      this.mediaUpdateTimeout = null;
    }
    const media = this.media();

    if (shouldDelay) {
      const delay = media ? ((media.partTargetDuration || media.targetDuration) / 2) * 1000 : 5 * 1000;

      this.mediaUpdateTimeout = window.setTimeout(() => {
        this.mediaUpdateTimeout = null;
        this.load();
      }, delay);

      return;
    }

    if (!this.started) {
      this.start(); // When onGroupChanged occurs, the this.start function is executed to get the new full segment and make the update work properly.
      return;
    }

    if (media && !media.endList) {
      this.trigger('mediaupdatetimeout');
    } else {
      this.trigger('loadedplaylist');
    }
  }

(playlist-loader.js)

Steps to reproduce

Explain in detail the exact steps necessary to reproduce the issue.

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a1",NAME="a0",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",LANGUAGE="ko",URI="https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/a0_a1/a1.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a2",NAME="a0",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",LANGUAGE="ko",URI="https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/a0_a2/a2.m3u8"
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1572000,BANDWIDTH=3144000,CODECS="avc3.640028,mp4a.40.2",RESOLUTION=1920x1080,AUDIO="a1"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v2/v2.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=822000,BANDWIDTH=1644000,CODECS="avc3.64001f,mp4a.40.2",RESOLUTION=1280x720,AUDIO="a1"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v4/v4.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=423000,BANDWIDTH=846000,CODECS="avc3.64001f,mp4a.40.2",RESOLUTION=854x480,AUDIO="a2"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v5/v5.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=235500,BANDWIDTH=471000,CODECS="avc3.64001e,mp4a.40.2",RESOLUTION=640x360,AUDIO="a2"
https://www.test.com/rl0g0whgef3hb7dquccx2bgtf/ll/v0_v7/v7.m3u8
  1. Play an m3u8 with two audio tracks in llhls.
  2. Video quality changes frequently.
  3. An audio playlist is working again after a long pause.

Results

Expected

Hopefully, the above issues have been resolved and LLHLS videos play smoothly despite frequent quality changes.

Error output

Uncaught TypeError: Cannot read properties of undefined (reading 'tracks')
    at Dh (video.min.js:38:91026)
    at i (video.min.js:38:94224)
    at Oh.i.activeXhrs (video.min.js:38:94670)
    at video.min.js:38:96100
    at Pd (video.min.js:37:20244)
    at Object.callback (video.min.js:37:20950)
    at n (video.min.js:12:44288)
    at XMLHttpRequest.e (video.min.js:12:45067)
    at XMLHttpRequest.sentryWrapped (helpers.ts:85:17)

Additional Information

Please include any additional information necessary here. Including the following:

videojs-http-streaming version

what version of videojs-http-streaming does this occur with?
videojs-http-streaming x.y.z

videojs version

what version of videojs does this occur with?
video.js x.y.z

Browsers

what browsers are affected? please include browser and version for each
*

Platforms

what platforms are affected? please include operating system and version or device and version for each
*

Other Plugins

are any other videojs plugins being used on the page? If so, please list them with version below.
*

Other JavaScript

are you using any other javascript libraries or frameworks on the page? if so please list them below.
*
@video-archivist-bot
Copy link

Hey! We've detected some video files in a comment on this issue. If you'd like to permanently archive these videos and tie them to this project, a maintainer of the project can reply to this issue with the following commands:

@welcome Welcome
Copy link

welcome bot commented Nov 15, 2023

👋 Thanks for opening your first issue here! 👋

If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.
To help make it easier for us to investigate your issue, please follow the contributing guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@windbella @video-archivist-bot