0.9.2

Released 2020-09-20

Bugfixes

• Optional[X] notation to mark a field as nullable never worked; it does now.
• __binobj_struct__.num_own_fields wasn't getting set for structs with their fields declared by assignment. As such, mixed field declarations (PEP 526 and assignment) silently passed, resulting in undefined behavior.
• MixedDeclarationsError is now thrown as expected when a struct mixes assignment and PEP 526 field declarations.
• typing.Union is now correctly rejected when a struct attempts to use it as if it were a binobj.Union. Using it will trigger a InvalidTypeAnnotationError as it was supposed to from the beginning.

Deprecations

pip has declared that Python 3.5 support will end January 2021. In keeping with the recently established compatibility rules, binobj will not make an effort to maintain Python 3.5 compatibility past then either.

Other Changes

Deleted some dead code.

0.9.1

Released 2020-09-08

Fixes installation problem on Python 3.5 due to a syntax error; otherwise identical to 0.9.0.

0.9.0

Released 2020-09-08

This is a significant release with an embarrassing number of bugfixes and a few new features enhancing field declarations, null value handling, and how absent fields are represented.

New Features

Dataclass Annotations

The most exciting feature in this release is the ability to use PEP 526 type annotations to declare fields on Python 3.6 and higher. Whereas before you had to assign class variables in the declarations, you can now do this:

@binobj.dataclass
class MyStruct(binobj.Struct):
    # Preferred: use a class object
    foo: UInt16

    # You can define default values like this
    bar: StringZ = ""

    # You can pass struct classes -- no need for a `Nested` wrapper. Forward
    # references using strings are *not* supported.
    sub_struct: MyOtherStruct

    # Instances are allowed but are less readable and will anger certain linters.
    # Be careful not to *assign* the field instance!
    baz: Timestamp64(signed=False)

    # You can pass functions for default values just as if you were calling the
    # constructor, but this looks confusing and is **not recommended**. This may
    # throw an exception in the future if I decide it's too egregious.
    bam: StringZ = lambda: os.sep

    # To make BinObj ignore a plain class variable, mark it with ClassVar.
    my_class_variable: ClassVar[int] = 123

There are a few restrictions:

• If you use the dataclass class decorator you must use PEP 526 type annotations for all fields in the struct.
• You can't use this on Python 3.5.

More flexible size

• All fields now accept a Field[int] object for the size argument in the constructor, as well as a string naming a field (useful for subclasses where the size field is in the superclass).
• A field whose size depends on another field can now use DEFAULT for null_value.

null_value doesn't need to be bytes

null_value now accepts deserialized values in addition to raw byte strings.

# This used to be your only option:
field = String(size=8, encoding="utf-16-le", null_value=b"N\x00U\x00L\x00L\x00")

# You now can do this as well:
field = String(size=8, encoding="utf-16-le", null_value="NULL")

New Argument: not_present_value

Instead of being hard-coded to return NOT_PRESENT when a field is missing, this new field argument allows returning a user-defined value. The default is still NOT_PRESENT.

>>> my_field = UInt8(not_present_value=-1, present=lambda *_a, **_k: False)
>>> my_field.from_bytes(b"")
-1

Miscellaneous

• The repr() of a Field now includes the field name.
• Added new exception CannotDetermineNullError.

Bugfixes

• When a Field throws ImmutableFieldError it now includes its name in the error message. It was accidentally omitted before.
• A variable-sized field using DEFAULT for its null value would crash with a TypeError upon serialization if it depended on another field for its size.
• Fixed wrong type annotation in size argument for Field and also its property.
• String didn't handle the case when its length was dictated by another field. It'd blow up with a TypeError when serializing. Deserializing worked, though.
• If null_value was DEFAULT the field would never load as None. This has been broken for a really long time.
• null_value when loading has been completely broken for quite some time; it now works for all fields except unsized ones such as StringZ.

Breaking Changes

Passing DEFAULT for null_value of an unsized field such as StringZ will throw a CannotDetermineNullError. This resolves the asymmetric behavior where using DEFAULT throws an error when dumping but erroneously tries to load whatever's next in the stream when loading, resulting in unpredictable behavior.

Deprecations

• Field._get_expected_size() has been made a public method. Use get_expected_size() instead. The private form will still work but is deprecated and will be removed in a future version.
• Support for Python 3.5 will be removed in the next non-patch release, either 0.10.0 or 1.0, whichever is next.

Other Changes

The .gitignore file now properly ignores autogenerated documentation files.

0.8.0

Released 2020-07-02

New Features

Official support for Python 3.9.

Bugfixes

• _do_load() could be given None for the loaded_fields argument even though the documentation explicitly stated that it was guaranteed to not be.
• _do_dump() would get given bytes as its value argument if the field's default value was None.
• The present callable was sometimes passed too few arguments, potentially resulting in a TypeError.
• Dumping an unsized iterable in an Array no longer crashes.
• Dumping a missing field whose default callable returns UNDEFINED now throws the expected MissingRequiredValueError exception instead of trying to serialize UNDEFINED.
• Test on PyPy 3.6 like we claimed we were. Accidentally deleted that in the travis.yml file.

Breaking Changes

• Removed the load(), loads(), dump(), and dumps() methods which were deprecated in 0.6.2.
• Array now skips over fields loading as NOT_PRESENT when loading.
• Field is now a generic container class, which means all subclasses must define their value type. This only affects users that created their own subclasses.

Other Changes

• PEP 484 type annotations have been added.
• Timestamp and its subclasses no longer inherit from Integer.
• _NamedSentinel has been eliminated. In keeping with PEP 484, sentinel values such as UNDEFINED and NOT_PRESENT are now enums. For more information on why, see Support for Singleton Types in Unions in the PEP 484 documentation.
• from binobj.errors import * now only imports the exception classes.
• Travis no longer supports PyPy 3.5 so we have to stop testing on it, but the tests pass on CPython 3.5 and PyPy 3.6 so I think you're okay for now.

0.7.1

Released: 2020-04-30

• __components__ and __validators__ were removed and consolidated into a single data structure called __binobj_struct__ with a stricter and more logical structure. This is a purely internal change and should not affect most users.
• Better documentation.

0.7.0

Released 2019-11-25

New Features

• Array now sets size if it's a fixed length and its components have fixed sizes as well. As a consequence, Struct.get_size() now returns a value if all arrays inside it are sized.
• Nested also sets size if the struct it wraps is of a fixed size.
• Struct.from_stream() and Struct.from_bytes() now support an additional argument, init_kwargs, that you can use to pass additional arguments to the struct's constructor. You can also use this to override a field's value.
• Struct now provides a repr that shows all of its values, e.g.

    MyStruct(foo=123, bar="456")

Bugfixes

Fixed URL typos in documentation.

Deprecations

Support for Python 3.5 is deprecated. According to the 3.5 release schedule, 3.5.9 was the last scheduled release on 2019-11-01.

Other Changes

• Now testing the released Python 3.8 version instead of the development version.
• Upgraded many testing dependencies.

0.6.6

Released 2019-11-25

Bugfixes

For some bizarre reason package detection from the setup.cfg file stopped working in January 2019 and every single release since 0.5.2 hasn't had the source code in it, and the wheels have been empty. In other words, you could install binobj but import binobj would fail!

This tweaks setup.py so that you can use it again.

0.6.5

Botched release, removed from PyPI.

0.6.4

• Add official support for PyPy 3.6
• Minor documentation tweaks.

0.6.3

Released: 2019-04-13

New Features

Add official support for Python 3.8.

Other Changes

• Minor documentation fixes.
• Convert entire repo to use black for code formatting. I don't agree with all of its opinions but I do think it's better to be consistent everywhere.