Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Node.js Buffers #302

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

MichaelBuhler
Copy link
Contributor

This PR adds the ability to construct GeoTIFF objects directly from Node.js Buffer data.

This PR implements the BufferSource class and exports the fromBuffer() function.


Currently, this library supports creating objects from ArrayBuffers. In Node.js, a Buffer is implemented as part of the address space of an underlying (possibly shared) ArrayBuffer. We can slice a new ArrayBuffer (which is affectively a view into the larger shared ArrayBuffer) which holds the relevant GeoTiff binary data. The new BufferSource class extends the existing ArrayBufferSource class.

This can be done is just one or two lines outside of this library, but it took some Googling and testing for me to figure out, so I think we can support Buffers directly with these simple convenience functions.

I also cleaned up some related documentation.

@constantinius
Copy link
Member

@MichaelBuhler Thanks for your contribution!

This introduces a breaking change by renaming makeBufferSource -> makeArrayBufferSource (which makes sense, but would break existing client code). So this has to go into a major release.

Other question: Isn't this essentially the same as using makeBufferSource(buffer.buffer) (the old one)? Or is the handling really that different?

@MichaelBuhler
Copy link
Contributor Author

MichaelBuhler commented Apr 21, 2022

I didn't think that makeBufferSource was exported from this package. I left the existing public interface untouched. I could refactor back to the old makeBufferSource() and make the new one makeNodeBufferSource() or similar. Let me know if you want that.


Other question: Isn't this essentially the same as using makeBufferSource(buffer.buffer) (the old one)? Or is the handling really that different?

Yes, it is. According to the Node.js docs for buffer.buffer here:

This ArrayBuffer is not guaranteed to correspond exactly to the original Buffer.

And continuing in the docs for buffer.byteOffset here:

...the buffer does not start from a zero offset on the underlying ArrayBuffer.

This can cause problems when accessing the underlying ArrayBuffer directly using buf.buffer, as other parts of the ArrayBuffer may be unrelated to the Buffer object itself.

There are several good answers on this StackOverflow question. Also see this gist comment.

It seems if the buffer is small enough it may quickly be allocated part of an existing ArrayBuffer memory pool.

On the other hand, for a sufficiently large Buffer, a new non-shared ArrayBuffer is allocated for it, so the underlying buffer.buffer works coincidentally, most of the time.

I don't know what is was about my buffers, but the way my application was using them, it didn't like using buffer.buffer directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants