Skip to content

v0.13.0 Release Plan & Test Matrix

John Bartos edited this page Sep 11, 2019 · 32 revisions

v0.13.0

New Features

Bugfixes

Misc. Improvements

v0.13.0 Manual Test Matrix

OS Browser Basic Playback 1 Live 2 Captions 3 ID3 4 Alt Audio 5 5
Desktop (MacOS/Windows) Firefox
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
Desktop (MacOS/Windows) Chrome
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
Android Chrome
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
Windows IE11
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
Windows Edge 18
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Assignee
  • Tests for specific features/fixes
  • Basic playback (mostly automated)
  • Live (not automated)
  • ID3
  • TS changes (could be covered by other suites)
  • Safari smoke test
  • Bundle analysis
  • Test with existing TS definitions

General:

Things to look out for:

  • Console warnings
  • Console errors
  • HLS errors
    • Especially internal exceptions

1 Test 1 Steps (Basic Playback):

  1. Begin playback
  2. Assert that playback begins at the configured start time (default 0)
    • Unless otherwise noted, Hls.js should not seek on startup (e.g. to skip a gap)
  3. Allow Hls.js to play for at least three segments
  4. Assert that playback continues smoothly and without issue
  5. Switch renditions
    • Test switching using both the Current Level and Next Level API
  6. Assert that Hls.js switches renditions smoothly and without issue
  7. Seek to at most three segments from the end
    • Unless otherwise noted, Hls.js should begin playback immediately (e.g. it should not try to skip a gap)
  8. Allow the stream to end
  9. Assert that Hls.js has signaled end of stream
    • video.ended should be true
    • BUFFER_EOS should have been fired
    • There should be no more network requests from Hls.js (check the Network tab)

2 Test 2 Steps (Live):

  1. Begin playback
  2. Assert that playback begins at the default time (default 3 segment lengths)
  3. Allow playback to continue for at least one minute
  4. Assert that playback continues smoothly and without issue
  5. Switch renditions
  6. Assert that Hls.js switches renditions smoothly and without issue
  7. Allow playback to continue for at least one minute
  8. Assert that playback continues smoothly and without issue
  9. Stop playback
  10. Assert that Hls.js does not download media while paused

3 Test 3 Steps (Captions):

  1. Begin playback
  2. Enable any captions track
  3. Assert that captions are displayed
  4. Allow playback to continue
  5. Assert that the correct cue is displayed at the corresponding time
    • VTT files can be inspected for the cue time
  6. Seek forwards
  7. Assert that the correct cue is displayed, and that the previous cue is no longer showing

4 Test 4 Steps (ID3):

  1. Set up Hls.js
  2. Attach a FRAG_PARSING_USERDATA to the Hls.js instance
  3. Begin playback

5 Test 5 Steps:

  1. Step 1
  2. Step 2