VLenUTF8().encode(buffer) fails is buffer is read-only #514
Comments
|
Does |
|
Do you mean like: Apparently not. I would have thought that this is handleable since |
|
Pandas has recently started wrapping the low-level arrays into immutable ones, which is maybe why you are seeing this now. I assume they internally access the low-level writable buffer somewhere. I think this is part of their move towards arrow, since arrow buffers are supposed to be immutable (which makes sense when there are offsets/indexes around, rather than just values). |
It looks like if you access the |
|
If you're not doing any Or we could require the caller to always provide a raw, writable numpy-like. |
|
My concern is that pandas may not intentionally be giving me a writable view, and may change this behaviour in the future. I was pointed at: For how pandas deals with this case. AFAICT, it's basically changing the typing from a |
|
I've opened a PR which should handle this on the numcodecs side. Does the approach look fine to you @martindurant? |
|
Yes, I suppose it's fine. We should maybe document this somewhere, since having to make a copy of the data, even temporarily, may surprise some people. |
Minimal, reproducible code sample, a copy-pastable example if possible
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[39], line 9 6 a = np.array(list("abc"), dtype=object) 7 a.flags.writeable = False ----> 9 codec.encode(a) File numcodecs/vlen.pyx:87, in numcodecs.vlen.VLenUTF8.encode() File <stringsource>:663, in View.MemoryView.memoryview_cwrapper() File <stringsource>:353, in View.MemoryView.memoryview.__cinit__() ValueError: buffer source array is read-onlyProblem description
Short description: this shouldn't error, as the codec shouldn't care whether it can write to the buffer it's passed.
Long description:
copy-on-write=Trueon import in the latest releaseI can't think of a reason that
.encodewould need to modify the buffer, so it shouldn't care that it's read-only.Version and installation information
Please provide the following:
numcodecs.__version__'0.12.1'Also, if you think it might be relevant, please provide the output from
pip listorconda listdepending on which was used to install NumCodecs.conda list
The text was updated successfully, but these errors were encountered: