feat: add version_matching module

[davigsousa]

As described at #220

I implemented an initial version of a version_matching module.

• As suggested by @leandromoreira, it is a separated module from parser.
• I added some version matching rules from the reference specified by @thenewguy. I hope we can add the following ones in next PRs.
• It only runs in strict mode

[davigsousa]

All errors are now raised when we finish to validate a file in strict mode, example:

Traceback (most recent call last):
  File "/home/..././invalid.py", line 4, in <module>
    content = m3u8.parse(invalid_versioned_playlists.M3U8_RULE_IV, strict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../m3u8/m3u8/parser.py", line 82, in parse
    raise Exception(found_errors)
Exception: [VersionMatchingError(line_number=2, line='#EXT-X-KEY: METHOD=AES-128, IV=0x123456789ABCDEF0123456789ABCDEF0, URI="https://example.com/key.bin"', how_to_fix='Change the protocol version to 2 or higher.', description='You must use at least protocol version 2 if you have IV in EXT-X-KEY.'), VersionMatchingError(line_number=4, line='#EXTINF: 10.0,', how_to_fix='Change the protocol version to 3 or higher.', description='You must use at least protocol version 3 if you have floating point EXTINF duration values.')]

I thought about printing them, but I didn't see any other prints in Parse, so I just raised

[leandromoreira]

Outstanding work you've done here, thanks a lot. I left some comments/nit but if you want, you can handle them in a new PR.

[leandromoreira]

For master variant is it mandatory to have version? if that's the case then I think we should return an error here stating that the master playlist doesn't contain a version.

Again we can leave that for a future PR if you wish.

[leandromoreira]

❤️

[leandromoreira]

nit: we should catch ValueError

[leandromoreira]

All errors are now raised when we finish to validate a file in strict mode, example:

Traceback (most recent call last):
  File "/home/..././invalid.py", line 4, in <module>
    content = m3u8.parse(invalid_versioned_playlists.M3U8_RULE_IV, strict=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/.../m3u8/m3u8/parser.py", line 82, in parse
    raise Exception(found_errors)
Exception: [VersionMatchingError(line_number=2, line='#EXT-X-KEY: METHOD=AES-128, IV=0x123456789ABCDEF0123456789ABCDEF0, URI="https://example.com/key.bin"', how_to_fix='Change the protocol version to 2 or higher.', description='You must use at least protocol version 2 if you have IV in EXT-X-KEY.'), VersionMatchingError(line_number=4, line='#EXTINF: 10.0,', how_to_fix='Change the protocol version to 3 or higher.', description='You must use at least protocol version 3 if you have floating point EXTINF duration values.')]

I thought about printing them, but I didn't see any other prints in Parse, so I just raised

I think raising it's the best we can do, I suggested we use the group exception so the natural print will be easier to read.

[davigsousa]

I suggested we use the group exception

As I said, I wanted to use ExceptionGroup, but it is from Python 3.11 and the project requires only Python3.7+, I dont know if we can use it here. 😕

[leandromoreira]

I suggested we use the group exception

As I said, I wanted to use ExceptionGroup, but it is from Python 3.11 and the project requires only Python3.7+, I dont know if we can use it here. 😕

that's okay, thanks for the efforts

[leandromoreira]

A minor feature we can add, if the verification list keeps growing, is a map/dict like structure to avoid N*N:

if hash(line) in verifications:
  verifications[verification_hash(line)].valid?

def verification_hash(line):
  if line.contains("version"):
    return "version"

[mauricioabreu]

Awesome work, @davigsousa

[mauricioabreu]

Could you check why it is failing for 3.7 version @davigsousa ?

[leandromoreira]

It looks like related to:

available_rules: List[type[VersionMatchRuleBase]] = []

Maybe replacing type[ by typing.Type or type(.

https://docs.python.org/3.7/library/typing.html?highlight=type#typing.Type

tests/test_invalid_versioned_playlists.py:9: in <module>
    import m3u8
m3u8/__init__.py:9: in <module>
    from m3u8.model import (
m3u8/model.py:8: in <module>
    from m3u8.parser import format_date_time, parse
m3u8/parser.py:17: in <module>
    from m3u8 import protocol, version_matching
m3u8/version_matching.py:4: in <module>
    from m3u8.version_matching_rules import VersionMatchingError, available_rules
m3u8/version_matching_rules.py:108: in <module>
    ValidEXTXBYTERANGEOrEXTXIFRAMESONLY,
E   TypeError: 'type' object is not subscriptable

[leandromoreira]

good work

[leandromoreira]

nit: maybe we should assert the exception or at least partially the error message/help

def test_raises():
    with pytest.raises(Exception) as exc_info:   
        raise Exception('some info')
    # these asserts are identical; you can use either one   
    assert exc_info.value.args[0] == 'some info'
    assert str(exc_info.value) == 'some info'

ref https://stackoverflow.com/a/23514853

[leandromoreira]

ref https://docs.pytest.org/en/7.1.x/how-to/assert.html

[leandromoreira]

ref https://pytest-with-eric.com/introduction/pytest-assert-exception/

[mauricioabreu]

Hello @davigsousa, are you still working on these awesome changes? If you don't have time to complete the task, I can take over the PR from here and handle the finalization and merging.

[davigsousa]

Hi @mauricioabreu , unfortunately I have very little time for open source lately, sorry for that! I wanted to finish the work here, but thank you for your help, I'll accept it! We could speed up the merge here.

You can take over, thank you very much!