Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable util::Result copying and assignment #29906

Merged
merged 2 commits into from Apr 30, 2024

Conversation

ryanofsky
Copy link
Contributor

This PR just contains the first two commits of #25665.

It disables copying of util::Result objects because unnecessary copies are inefficient and not possible after #25665, which makes util::Result object move-only.

It disables the assignment operator and replaces it with an Update() method, because #25665 adds more information to util::Result objects (warning and error messages and failure values) and having an assignment operator that overwrites data instead of merging it would make it easy to accidentally erase existing information while trying to assign new information.

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 18, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK stickies-v, furszy, achow101
Stale ACK TheCharlatan

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29936 (fuzz: wallet: add target for CreateTransaction by brunoerg)
  • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)
  • #29442 (wallet: Target a pre-defined utxo set composition by adjusting change outputs by remyers)
  • #28984 (Cluster size 2 package rbf by instagibbs)
  • #28676 ([WIP] Cluster mempool implementation by sdaftuar)
  • #26022 (Add util::ResultPtr class by ryanofsky)
  • #25722 (refactor: Use util::Result class for wallet loading by ryanofsky)
  • #25665 (refactor: Add util::Result failure values, multiple error and warning messages by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, breaking it out in a smaller PR seems like a good idea, thanks.

I'm unsure about removing the assignment operator altogether. I think it's good to be careful about footguns, but I'm worried about making the interface overly unintuitive.

Would it make sense to only merge another Result when using the Merge (currently called Update) method, and use the assignment operator to overwrite the current result? I think when people want to merge results, they'll probably want to be explicit about that, and in other cases overwriting/clearing everything seems like expected behaviour?

src/wallet/test/fuzz/coinselection.cpp Show resolved Hide resolved
src/wallet/spend.cpp Show resolved Hide resolved
@@ -75,6 +85,9 @@ class Result
const T& operator*() const LIFETIMEBOUND { return value(); }
T* operator->() LIFETIMEBOUND { return &value(); }
T& operator*() LIFETIMEBOUND { return value(); }

//! Update this result by moving from another result object.
Result& Update(Result&& other) LIFETIMEBOUND { return *this = std::move(other); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update is a bit of an ambiguous name imo, it doesn't clearly indicate if it means replacing or merging into the existing result. I think for this commit, where we can only overwrite the existing result, I think Replace would be better. For the next commits in #25665 where we actually add the merging functionality, I think Merge or Combine would be a better choice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #29906 (comment)

Update is a bit of an ambiguous name imo, it doesn't clearly indicate if it means replacing or merging into the existing result. I think for this commit, where we can only overwrite the existing result, I think Replace would be better. For the next commits in #25665 where we actually add the merging functionality, I think Merge or Combine would be a better choice.

We could definitely have a Replace method in the future. The only reason I didn't add one now is that so far after updating a lot of code to use the Result class in PR's #29700 and #25722, I haven't seen uses for a Replace method. If you think having a Replace() method would good to add here for a future use-case, or for the sake of completeness, I'd be happy to add it. However, I don't think any changed lines here should be calling Replace() because they are just trying to update result values, not reset and replace the entire result object.

I did originally consider calling this method Merge instead of Update but I liked Update better for 2 reasons. First, I liked consistency with python's dict.update method which takes fields from two different dict objects and combines them, just like this Update function does. Second, I think the name Merge would be misleading in the majority of cases where individual values are not merged, only different fields are combined, and new field values overwrite old values. 1

Your suggestion Combine could also work instead of Update but I think is less clear syntactically. For example, I think a.Update(b) maps pretty well to "update object a with contents of object b", but it's less obvious what a.Combine(b) would be supposed to do.

Footnotes

  1. It is true that with #25722, you can go out of your way to pass a custom type to util::Result and specialize ResultTraits so field values are merged when Update() is called. I haven't seen cases where this is useful for success types, only for some failure types where multiple things can fail and you want to return information about all the failures. I think name Update is generic enough to cover atypical cases where values are merged without implying that values are merged typically.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen uses for a Replace method.

I agree. The use cases I was thinking of should just be addressed with list initialization, so I think not implementing the method until we actually need it is the way to go, thanks.

Second, I think the name Merge would be misleading ... only different fields are combined

That's a good point, I agree Update > Merge

it's less obvious what a.Combine(b) would be supposed to do

I don't think I agree, but I do agree there's no clear winner, so I'm happy to leave this as is.

Copy link
Contributor Author

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 974b16b -> c45cb13 (pr/saferesult.1 -> pr/saferesult.2, compare) with suggestions

re: #29906 (review)

I'm unsure about removing the assignment operator altogether. I think it's good to be careful about footguns, but I'm worried about making the interface overly unintuitive.

We could add back a safer, more limited form of operator= in the future if not having it causes friction. I rewrote commit message to be clearer and mention that. But I think having a named Update() method should make it easier to figure out what code is doing and avoid mistakes.

Would it make sense to only merge another Result when using the Merge (currently called Update) method, and use the assignment operator to overwrite the current result? I think when people want to merge results, they'll probably want to be explicit about that, and in other cases overwriting/clearing everything seems like expected behaviour?

Hmm, I'm actually not sure what this is suggesting. If there is a need to overwrite/clear results, IMO, the best way to do that would be to add a Reset() method. I haven't seen an actual need to do that anywhere though. The intended use for Update is to be able to write code like:

util::Result<SuccessType, FailureType> result;
if (do_something()) {
  result.Update(success_value);
} else {
  result.Update({util::Error{_("error message"), failure_value});
}

@@ -75,6 +85,9 @@ class Result
const T& operator*() const LIFETIMEBOUND { return value(); }
T* operator->() LIFETIMEBOUND { return &value(); }
T& operator*() LIFETIMEBOUND { return value(); }

//! Update this result by moving from another result object.
Result& Update(Result&& other) LIFETIMEBOUND { return *this = std::move(other); }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #29906 (comment)

Update is a bit of an ambiguous name imo, it doesn't clearly indicate if it means replacing or merging into the existing result. I think for this commit, where we can only overwrite the existing result, I think Replace would be better. For the next commits in #25665 where we actually add the merging functionality, I think Merge or Combine would be a better choice.

We could definitely have a Replace method in the future. The only reason I didn't add one now is that so far after updating a lot of code to use the Result class in PR's #29700 and #25722, I haven't seen uses for a Replace method. If you think having a Replace() method would good to add here for a future use-case, or for the sake of completeness, I'd be happy to add it. However, I don't think any changed lines here should be calling Replace() because they are just trying to update result values, not reset and replace the entire result object.

I did originally consider calling this method Merge instead of Update but I liked Update better for 2 reasons. First, I liked consistency with python's dict.update method which takes fields from two different dict objects and combines them, just like this Update function does. Second, I think the name Merge would be misleading in the majority of cases where individual values are not merged, only different fields are combined, and new field values overwrite old values. 1

Your suggestion Combine could also work instead of Update but I think is less clear syntactically. For example, I think a.Update(b) maps pretty well to "update object a with contents of object b", but it's less obvious what a.Combine(b) would be supposed to do.

Footnotes

  1. It is true that with #25722, you can go out of your way to pass a custom type to util::Result and specialize ResultTraits so field values are merged when Update() is called. I haven't seen cases where this is useful for success types, only for some failure types where multiple things can fail and you want to return information about all the failures. I think name Update is generic enough to cover atypical cases where values are merged without implying that values are merged typically.

src/wallet/spend.cpp Show resolved Hide resolved
src/wallet/test/fuzz/coinselection.cpp Show resolved Hide resolved
Copy link
Contributor Author

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated c45cb13 -> 6602231 (pr/saferesult.2 -> pr/saferesult.3, compare) adding change to actually disable the copy constructor

src/wallet/test/fuzz/coinselection.cpp Show resolved Hide resolved
Copy link
Contributor

@TheCharlatan TheCharlatan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 6602231

Copy link
Member

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 6602231

src/util/result.h Outdated Show resolved Hide resolved
Comment on lines 380 to 385
op_dest.Update(walletModel->wallet().getNewDestination(address_type, strLabel));
if (!op_dest) {
editStatus = KEY_GENERATION_FAILURE;
return QString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR but.. we could remove this Update call pretty easily (e8a3305). We are calling getNewDestination twice only to verify if the wallet is locked or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re: #29906 (comment)

It seems like the goal of the code is to get a new address without unlocking. But I haven't looked into this. It definitely looks a little clumsy, though, because it isn't actually using the error message that is returned.

ryanofsky and others added 2 commits April 25, 2024 16:08
`util::Result` objects are aggregates that can hold multiple fields with
different information. Currently Result objects can only hold a success value
of an arbitrary type or a single bilingual_str error message. In followup PR
bitcoin#25722, Result objects may be able to
hold both success and failure values of different types, plus error and warning
messages.

Having a Result::operator= assignment operator that completely erases all
existing Result information before assigning new information is potentially
dangerous in this case. For example, code that looks like it is assigning a
warning value could erase previously-assigned success or failure values.
Conversely, code that looks like it is just assigning a success or failure
value could erase previously assigned error and warning messages.

To prevent potential bugs like this, disable Result::operator= assignment
operator.

It is possible in the future we may want to re-enable operator= in limited
cases (such as when implicit conversions are not used) or add a Replace() or
Reset() method that mimicks default operator= behavior. Followup PR
bitcoin#25722 also adds a Result::Update()
method providing another way to update an existing Result object.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
Copying util::Result values is less efficient than moving them because they
allocate memory and contain strings. Also this is needed to avoid compile
errors in bitcoin#25722 which adds a
std::unique_ptr member to util::Result which implicity disables copying.
Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK, code LGTM 6602231 with some non-blocking suggestions.

We could add back a safer, more limited form of operator= in the future if not having it causes friction.

I made my previous comment because I thought there are some instances of somewhat awkward Update usage in this PR. The majority of util::Result usage is/should probably not require Update() at all, because we're just returning either a success value or an error string. But, in those simple cases, we should just use list initialization, imo, and not assignment initialization. So, I've left a couple of code comments that remove all the instances I find awkward.

The intended use for Update is to be able to write code like: ...

Hmmm. In the code example you give, I'd find assignment much more intuitive, since the goal doesn't seem to be combining results. If Update() is aimed to be used exclusively (?) when we intend to combine results, doesn't that make for a more clear interface?

src/init.cpp Outdated Show resolved Hide resolved
src/test/mempool_tests.cpp Outdated Show resolved Hide resolved
src/wallet/test/fuzz/notifications.cpp Outdated Show resolved Hide resolved
src/wallet/test/spend_tests.cpp Outdated Show resolved Hide resolved
@@ -75,6 +85,9 @@ class Result
const T& operator*() const LIFETIMEBOUND { return value(); }
T* operator->() LIFETIMEBOUND { return &value(); }
T& operator*() LIFETIMEBOUND { return value(); }

//! Update this result by moving from another result object.
Result& Update(Result&& other) LIFETIMEBOUND { return *this = std::move(other); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen uses for a Replace method.

I agree. The use cases I was thinking of should just be addressed with list initialization, so I think not implementing the method until we actually need it is the way to go, thanks.

Second, I think the name Merge would be misleading ... only different fields are combined

That's a good point, I agree Update > Merge

it's less obvious what a.Combine(b) would be supposed to do

I don't think I agree, but I do agree there's no clear winner, so I'm happy to leave this as is.

@ryanofsky
Copy link
Contributor Author

Thanks @stickies-v, I agree with your suggestions to avoid updating existing Result objects where possible and will apply them.

But I wonder if there are is anything else I can do to help make the interface more intuitive though. I'm not exactly clear on why it is not intuitive currently. A few places you mentioned "Update here is confusing" but I don't understand why it is confusing. Saying result.Update(success_value) is just meant to update the result object with a success value while keeping other result data intact. Saying result.Update({util::Error{_("error message"), failure_value}); is meant to update the result object with an error message and failure value, again keeping other result data intact.

I think it's very similar to python dict.update function where if you start with an existing dictionary:

d = {'a': 1, 'b': 2, 'c': 3}

and call the update method with new values:

d.update({'a': 4, 'd': 5})

And the dictionary will be updated with the new values overwriting the old ones:

{'a': 4, 'b': 2, 'c': 3, 'd': 5}

In the code example you give, I'd find assignment much more intuitive, since the goal doesn't seem to be combining results. If Update() is aimed to be used exclusively (?) when we intend to combine results, doesn't that make for a more clear interface?

Update is just intended to update some data in a result object without necessarily erasing other data in the result object. Assigning is conceptually different and would imply that result being assigned to would be cleared and completely replaced by value being assigned. In this PR, the result object only has two mutually exclusive fields (success value and error string), so assigning and updating are the same thing. In #25665, result objects contain more fields so assigning and updating are different things.

Copy link
Contributor Author

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 6602231 -> 4fb08c2 (pr/saferesult.3 -> pr/saferesult.4, compare) with suggested changes using new result objects to avoid updating existing ones where possible
Update 4fb08c2 -> 88aa093 (pr/saferesult.4 -> pr/saferesult.5, compare) with no changes overall, just moving changes out of first commit that belonged in second commit

src/init.cpp Outdated Show resolved Hide resolved
src/test/mempool_tests.cpp Outdated Show resolved Hide resolved
src/wallet/test/fuzz/notifications.cpp Outdated Show resolved Hide resolved
src/wallet/test/spend_tests.cpp Outdated Show resolved Hide resolved
@stickies-v
Copy link
Contributor

Sorry, I've been doing a poor job explaining myself. I am fully onboard with how Update works. When I used the word "confusion", I was talking about usage, not interface*. Prior to this push, Update() was used in a couple of places where we know or expect the Result to be empty, and in those case I think we should just be explicit about that and use list initialization. When I see result.Update() being called, I have to look up which values are in result already and ensure that's correct, whereas with auto result{...}; I know right away that we're starting from an empty result.

To stick with the python analogy: using update when we know or expect a to be an empty dictionary, is imo an anti-pattern and should be avoided.

a = {}
// do some logic to calculate `value`
a.update({1: value})

Instead, it is much clearer to use assignment, so the reader doesn't have to figure out which values a held already.

// do some logic to calculate `value`
a = {1: value}

In short: I think it would be helpful if we only use result.Update() when result (potentially) has had fields set already, and use list initialization (or if necessary assignment initialization) for when we know (and thus enforce that) result is empty. This is also the concern I had with the code example at the end of this comment.

I hope I did a better job explaining myself, but otherwise I'm happy to talk about it more offline to not pollute this PR more.

  • well, except for making sure the method names align with how we intend people to use them, hence my earlier renaming suggestions.

@ryanofsky
Copy link
Contributor Author

ryanofsky commented Apr 26, 2024

Rebased 88aa093 -> 0cbcb66 (pr/saferesult.5 -> pr/saferesult.6, compare) dropping Result::Update() method in this PR, and last 2 uses of it.
Update 0cbcb66 -> e1be443 (pr/saferesult.6 -> pr/saferesult.7, compare) just changing a commit message to no longer mention Result::Update().


re: #29906 (comment)

Thanks for clarifying. I think we might disagree whether certain uses of an update function would be antipatterns, but I completely agree with all the code changes you suggested, and that in general if you have a choice between initializing values once or letting them change over time, you should prefer initializing them once so there's less changing state to reason about. I also looked at remaining two update calls in this PR, and thought they both were not good uses, so I removed them as well.

My update example was very simple because I wanted to show minimal code that demonstrated how to use the update function, but the code was probably too simple to demonstrate why you would want to use it. The use-case for update is when you have a function that is performing multiple operations and you want to accumulate warnings and errors from all the operations and return them. Examples would be the BlockManager::SaveBlockToDisk function from #29700 or the CompleteChainstateInitialization function from #25665. I think these are reasonable uses of the update function, but it's possible there could be better ways to write this code, and you might have another idea what it should look like.

Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK e1be443

The use-case for update is when you have a function that is performing multiple operations and you want to accumulate warnings and errors from all the operations and return them.

I agree, and from what I've seen so far, I would prefer that to be the only use-case.

I also looked at remaining two update calls in this PR, and thought they both were not good uses, so I removed them as well.

Ah, nice one. Given that chaining results is not actually implemented yet, it makes sense that there should be no existing usage of Update in these first commits. I looked at the initial implementation of these 2 new updates to make sure we're not missing anything and indeed, chaining is not relevant.


I've posted my response to the second part of your comment about Update() usage on #25665, since Update() is no longer in this PR.

src/test/mempool_tests.cpp Show resolved Hide resolved
src/validation.cpp Outdated Show resolved Hide resolved
@achow101
Copy link
Member

ACK e1be443

src/util/result.h Outdated Show resolved Hide resolved
Copy link
Contributor Author

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated e1be443 -> 9552619 (pr/saferesult.7 -> pr/saferesult.8, compare) with small suggested cleanups

src/test/mempool_tests.cpp Show resolved Hide resolved
src/util/result.h Outdated Show resolved Hide resolved
src/validation.cpp Outdated Show resolved Hide resolved
src/wallet/test/fuzz/notifications.cpp Outdated Show resolved Hide resolved
@stickies-v
Copy link
Contributor

Re-ACK 9552619 , just addressing nits, no other changes

@DrahtBot DrahtBot requested a review from achow101 April 27, 2024 14:12
Copy link
Member

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 9552619

Copy link
Contributor

@TheCharlatan TheCharlatan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 9552619

Copy link
Contributor Author

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated 9552619 -> 6a8b2be (pr/saferesult.8 -> pr/saferesult.9, compare) tweaking variables to avoid shadowing

src/validation.cpp Outdated Show resolved Hide resolved
@stickies-v
Copy link
Contributor

re-ACK 6a8b2be

Copy link
Member

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK 6a8b2be

@achow101
Copy link
Member

ACK 6a8b2be

@achow101 achow101 merged commit 2d30567 into bitcoin:master Apr 30, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants