-
Notifications
You must be signed in to change notification settings - Fork 35.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
assumeutxo, rpc: Improve EOF error when reading snapshot metadata in loadtxoutset #28670
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
65ef0d8
to
acb4d35
Compare
3e013e2
to
9a56b40
Compare
Rebased + fix correct usage of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to add test coverage for this error.
Also, the "file size being too small" error message isn't really descriptive and it does not provide further guidance to solve the problem. The file could be corrupt or could had been crafted erroneously.
I would go simpler for now and change "file size being too small" for "Unable to load UTXO snapshot, the file %s cannot be parsed."
I see your alternative solution better. More comprehensive.
45b005c
to
2b6d44a
Compare
Updates:
|
54e0121
to
4d175c1
Compare
4d175c1
to
22e4442
Compare
🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the Possibly this is due to a silent merge conflict (the changes in this pull request being Leave a comment here, if you need help tracking down a confusing failure. |
22e4442
to
50c7a76
Compare
50c7a76
to
53fbf19
Compare
Addressed @hebasto's suggestion in order to fix the MSVC CI job. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK
Code review ACK 53fbf19 |
🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the Possibly this is due to a silent merge conflict (the changes in this pull request being Leave a comment here, if you need help tracking down a confusing failure. |
Updates:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The CI failure is due a merge conflict. Need to rebase the PR. Some of the code you are using here is not in master anymore.
de514b9
to
521de52
Compare
Rebased. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re-ACK 521de52
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK 521de52
In the RPC command `loadtxoutset`, catch possible errors while reading snapshot metadata, most commonly due to the structure size being shorter than the expected one (40 bytes) or the snapshot file may be corrupted or crafted in an incompatible format.
Add a test to cover scenarios where an EOF error is raised during the reading of invalid snapshot metadata (40 bytes file size min) or a corrupted file with an invalid format.
521de52
to
4a2df05
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK 4a2df05
// Handle any exception, which includes reaching the end of the file e.g. file size < sizeOf(metadata) | ||
throw JSONRPCError(RPC_DESERIALIZATION_ERROR, strprintf("Unable to load UTXO snapshot, " | ||
"couldn't read snapshot metadata from file %s.\nThe file may be corrupted " | ||
"or it was crafted in an incompatible format.", path.utf8string())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: should use fs::PathToString(path)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PathToString
That is only for internal use. However, the result here will be returned on RPC, so the dev-notes apply:
- Use `fs::path::u8string()`/`fs::path::utf8string()` and `fs::u8path()` functions when converting path
to JSON strings, not `fs::PathToString` and `fs::PathFromString`
- *Rationale*: JSON strings are Unicode strings, not byte strings, and
RFC8259 requires JSON to be encoded as UTF-8.
Hm, unfortunately, I think this PR was made redundant by #29612 recently. After the latest feedback there which I addressed just 3 weeks ago, we are enhancing the assumeutxo metadata there significantly and this means all of the lines changed here are touched there as well and I don't see much of an overlap either that would let me use the code here. |
No prob, thanks for letting me know, I'll review that PR. |
Improve error messaging in
loadtxoutset
to provide a more user-friendly error message when the utxo snapshot file size is insufficient to contain the necessary metadata structure (40 bytes).Original strategy was to check the size of the file before de-serialisation and the current strategy was presented at that time as an alternative which has been preferred by reviewers and it looks simpler since
streams
source code files are no longer touched.This PR needs to be backported to 26.x