New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inconsistent output with different nbWorkers values #31
Comments
I just found this which is the exact same thing I observed: This seems like a minor thing, but for my particular use case, consistently reproducible archives are important. This appears to be the only issue with consistency I have found. (The multi-threading works wonderful by the way!) |
Could you provide code and input samples? |
Attached is a sample program that demonstrates the original problem as described above. Quick solution was to remove the Flush(), and just keep the Close(). (Flush is not needed anyway, as far as I can tell.)
So may well be a non-issue, just need to know not to call Flush before close. (I can provide a link to the test file we are using, if you want to look at this any further.) Thanks |
Changed |
@johnsanc314 @gjefferyes |
So I can confirm that with the latest changes if I call flush() before close() then I always get the 01 00 00 on the end of the stream. If I call zOut.Flush() followed by zOut.Close() I always now get 01 00 00 I guess my next question would be which is 'correct' should a zstd stream have the 01 00 00 on the end of it, or is this not needed? (It would appear that the 7z zstd fork does not have 01 00 00 on the end of its streams) |
Both situations are correct, if at |
I noticed that different nbWorkers parameter values can result in slightly different output for the compressed data.
I cannot find a pattern to when this happens, but when it does, it appears to be consistent for that number of nbWorkers for a particular input. It looks like an additional 3 bytes of "01 00 00" is added at the very end of the compressed stream. This is always the only difference. In my testing thus far, this extra 3 bytes appears to only occur with higher nbWorker values (22+), but some other higher values also omit these 3 bytes.
Any idea why this might be happening?
Example: 12 vs 32 nbWorkers for compressing an ISO file that is originally 3,469,070,336 bytes, packed into a zip archive:
The text was updated successfully, but these errors were encountered: