New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"error in roaringArray.readFrom: did not find expected serialCookie in header" when reading a bitmap written by roaring64 #409
Comments
Note: sorting the input file seems to make replication way faster. Also, here's a smaller test case (also found by @wjohnson-aurora) |
The bug is absolutely real. The issue is that when deserializing a 64-bit roaring bitmap, for some reason, the code first tries to deserialize a 32-bit version. I don't know why it is done, but in the instances you have created, it gets confused. It thinks it is dealing with a 32-bit bitmap, and then everything breaks after that. (Of course, it is not, you serialize a 64-bit roaring bitmap.) |
…to deserialize a 32-bit bitmap. It would usually work, but sometimes it would fail (see #409) because the number of containers would be recognized as a cookie. This PR just disables this attempt at supporting 32-bit bitmaps loading from 64-bit bitmaps.
Feel free to review my potential fix at #410 Note that the data is not corrupted or any such thing. It is just that the code gets confused at the deserialization stage. |
In fact, a review would be much appreciated. |
Thank you for fixing this so quickly! I can confirm that commit |
We've occasionally seen the following error when using
roaring64.Bitmap.ReadFrom
to read data written byroaring64.Bitmap.WriteTo
:I was able to find random data that replicates the error. To replicate:
uint64
s): roaring_error_items.txtmain.go
containing the following code:go.mod
with the following contents:go mod tidy
ReadFrom
:The text was updated successfully, but these errors were encountered: