Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable padding #318

Open
pohlt opened this issue May 2, 2022 · 3 comments
Open

Enable padding #318

pohlt opened this issue May 2, 2022 · 3 comments

Comments

@pohlt
Copy link

pohlt commented May 2, 2022

I have a use case where I send a few values and a chuck of bytes to JavaScript using msgpack. Most libraries (including msgpackr) try to unpack the data with as little copying as possible. Therefore, the chunk of bytes is unpacked as a Uint8Array which actually is just a view of the entire message with an offset (I could ask msgpackr to copy the data into a newly allocated array, but I try to avoid copying).

The chuck of bytes are actually 16-bit values, so I want convert it (still using the same original data) using new Uint16Array(msg.data.buffer, msg.data.byteOffset).

However, this doesn't work for my message, because the offset happens to be odd in this specific message and JS requires it to be divisible by 2 (as I'm asking for 16-bit values). I manually introduced new items in my dictionary where I tweaked the string lengths to shift the chunk of bytes to an appropriate alignment, but that feels wrong.

Does the msgpack format have something like a padding value (ideally a single byte) which could by used by packers to shift data to preferred byte alignments?

I know that msgpack is all about the smallest message size, but being able to add a single byte (or two or three) could save you a lot of copying on the receiver side.

@methane
Copy link
Member

methane commented May 2, 2022

No

@mlsomers
Copy link
Contributor

mlsomers commented May 2, 2022

No official padding support but you could misuse Nil (0xc0) for that purpose.
Be sure to document it well if any 3rd party is to be consuming or providing the content :-)

@pohlt
Copy link
Author

pohlt commented May 3, 2022

Wouldn't a Nil (0xc0) be interpreted as a regular none/null/None/nil value in an array or map?

How about 0xc1? To make it work, msgpack would need to specify that it has to be ignored independent of its context. Is that realistic?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants