Simplify coin selection for sendcoins #8516

Chinwendu20 · 2024-03-04T05:51:04Z

Change Description

This PR fixes this issue: #6949 (comment)
Depends on: btcsuite/btcwallet#912 and #8729

In this well detailed issue comment, the process of sending funds using utxos selected by the sender currently has over five steps in which one has to call various psbt APIs.

The goal of this PR is to improve user experience by including utxos to the sendcoins command and corresponding rpc structs (SendCoinsRequest and SendCoinsResponse ) which would enable the sender described to achieve the same aim with one command.

This is done by adding a new field to rpc structs SendCoinsRequest and SendCoinsResponse to enable users request for that functionality and enable the driving function (in which we modify its functionality as well) use these utxos when crafting the transaction for this functionality.

Additionally, the sweepall field in the above mentioned request and response rpc structs when true now not only sweeps ALL funds in the wallet but also ALL funds in the selected utxos when used in conjunction with the new select utxos field.

The lncli sendcoins command was updated to include the flag, utxo to enable this functionality on that end.

There were also slight internal logic change, where a slice of utxos are now accepted by relevant functions as variadic argument, functional options to enable this functionality.

Steps to Test

Steps for reviewers to follow to test the change.

Pull Request Checklist

Testing

Your PR passes all CI checks.
Tests covering the positive and negative (error paths) are included.
Bug fixes contain tests triggering the bug to prevent regressions.

Code Style and Documentation

The change obeys the Code Documentation and Commenting guidelines, and lines wrap at 80.
Commits follow the Ideal Git Commit Structure.
Any new logging statements use an appropriate subsystem and logging level.
Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
There is a change description in the release notes, or [skip ci] in the commit message for small changes.

📝 Please see our Contribution Guidelines for further guidance.

coderabbitai · 2024-03-04T05:51:08Z

Important

Review Skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to full the review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Chinwendu20 · 2024-03-04T06:40:35Z

Hello @hieblmi here is the PR

sweep/walletsweep.go

Chinwendu20 · 2024-03-04T11:07:14Z

I would be using the psbt API instead as this approach would not craft a transaction with all inputs selected by the user as pointed out here:
btcsuite/btcwallet#912 (comment)

Modified it, no need for this

itest/lnd_channel_funding_fund_max_test.go

Chinwendu20 · 2024-03-05T06:25:44Z

Hello @sputn1ck, do you think I should create a different function instead of adding an arg to an exported one as per your comment here: btcsuite/btcwallet#912 (comment)

bennyhodl · 2024-03-12T18:59:31Z

tACK.

Tested locally and worked as expected with one utxo, sweep, and multiple utxos. Would like to see this merged as I am signing PSBTs manually because my application requires 1 input --> 1 output transactions.

bennyhodl · 2024-03-26T14:42:51Z

Is there an update on this @Chinwendu20 ?

Chinwendu20 · 2024-03-26T14:44:14Z

Is there an update on this @Chinwendu20 ?

Thanks I should push an update by the end of this week..

hieblmi

Great work once again, got a small Q regarding the added itest and a UX issue I found.

hieblmi · 2024-05-13T10:38:35Z

sweep/walletsweep.go

+func fetchUtxosFromOutpoints(utxos []*lnwallet.Utxo,
+	outpoints []wire.OutPoint) ([]*lnwallet.Utxo, error) {
+
+	lookup := fn.SliceToMap(utxos, func(utxo *lnwallet.Utxo) wire.OutPoint {


hieblmi · 2024-05-13T10:49:02Z

itest/lnd_sendcoins_utxo_test.go

+			name: "sendCoins, amount specified, " +
+				"no select outpoints",
+			amt: 200_000,
+		},


Are you referring to this test "sendCoins with selected utxos, expect change"?

I was thinking of something like

{ selectedCoins: []btcutil.Amount{ 50_000, 20_000, 1_000, }, amt: 70_100, ..............., },

hieblmi · 2024-05-13T11:13:20Z

cmd/lncli/commands.go

@@ -330,16 +331,28 @@ var sendCoinsCommand = cli.Command{
 				"scripts",
 		},
 		coinSelectionStrategyFlag,
+		cli.StringSliceFlag{
+			Name: "utxo",


If we pass in the same utxo twice the command isn't failing which I think should happen from a UX perspective. I tested(last utxo is a dupe): reg_zane sendcoins --addr bcrt1pztmhas8h2vg8xfvj4kr4s3kgx69d4usq6uxxmyh6lhslrgllwpns87snxq --utxo 8c787614dc1adfda9d09f5a908853d8e91cc5579bc00e43c3958c5cb7dcfaf46:0 --utxo a2a9f76c008f4fecdcfe34fc93da1a5f4f7e63a1f896dec9869272b101ac89a9:1 --utxo a2a9f76c008f4fecdcfe34fc93da1a5f4f7e63a1f896dec9869272b101ac89a9:1 --sweepall

Nice catch! I would fix it

Chinwendu20 · 2024-05-14T08:52:07Z

This PR is now dependent on #8750 and btcsuite/btcwallet#928

lightninglabs-deploy · 2024-05-21T08:55:37Z

@sputn1ck: review reminder
@hieblmi: review reminder

hieblmi · 2024-05-24T11:59:21Z

There are still itests failing.

Chinwendu20 · 2024-05-24T14:11:48Z

I think the taproot test needs to create a new node and not use the general Alice node. Two tests can request sendCoins to sweep all coins at the same time, creating the same transaction same txid, the other test can get the txid first into the mempool making the sweepall sendcoins operation fail in the other. I do not know if that makes sense. If it does I can open a new PR for a fix.

In the meantime, this needs rebasing but I need to know where this stands in approval so I make changes at once.

Chinwendu20 · 2024-05-24T14:15:08Z

Maybe all tests should create their own node to prevent cases like this?

hieblmi · 2024-05-27T09:27:01Z

I think the taproot test needs to create a new node and not use the general Alice node. Two tests can request sendCoins to sweep all coins at the same time, creating the same transaction same txid, the other test can get the txid first into the mempool making the sweepall sendcoins operation fail in the other. I do not know if that makes sense.

Not sure how this applies to the tests, in here lnd_taproot_test.go:128 sendcoins is called sequentially, why is the last call exceeding its deadline?

I think we should fix the itests in a separate commit in this PR.

Chinwendu20 · 2024-05-27T11:53:52Z

When I checked the logs for Alice related to this test, I saw this

2024-05-24 09:38:40.442 [WRN] BTWL: Transaction 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d not accepted by mempool: txn-already-in-mempool
2024-05-24 09:38:40.592 [INF] LNWL: Inserting unconfirmed transaction 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d
2024-05-24 09:38:40.943 [DBG] LNWL: Marked address bcrt1pn388y03pjsedvrl2n0dxgyz7yup9q6snhyrqj67qh2aj8dq88udq6wfs78 used
2024-05-24 09:38:41.462 [INF] RPCS: [sendcoins] spend generated txid: 8b7ab6dc1ace16e59491caed739a599756967cc23ea799f2f48bdd3d61116f8d

sendcoins is called sequentially in that test but other tests using the Alice node would call sendcoins in parallel, right?
So my initial thinking is that because the tx failed the TestMempool check and also that it would return an error while publishing the transaction it would time out with that error but the last log line shows that infact the the sendcoins rpc function exits successfully in spite of that and following the code it makes sense because ErrTxAlreadyInMempool is usually ignored.

Maybe if we rerun this it would pass, seems like a case of unprecedented slow server response

Chinwendu20 · 2024-05-28T09:15:14Z

fyi I ran this test locally and it was successful.

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Chinwendu20 requested review from yyforyongyu, ziggie1984, Roasbeef, Crypt-iQ and ellemouton as code owners March 4, 2024 05:51

Chinwendu20 mentioned this pull request Mar 4, 2024

Use selected Utxos while crafting txn btcsuite/btcwallet#912

Merged

Chinwendu20 force-pushed the sendcoins branch from 5b80b46 to 7f7b064 Compare March 4, 2024 06:15

hieblmi assigned Chinwendu20 Mar 4, 2024

guggero requested review from guggero and removed request for Roasbeef, yyforyongyu and ziggie1984 March 4, 2024 08:44

Chinwendu20 commented Mar 4, 2024

View reviewed changes

sweep/walletsweep.go Outdated Show resolved Hide resolved

Chinwendu20 marked this pull request as draft March 4, 2024 11:05

Chinwendu20 commented Mar 5, 2024

View reviewed changes

itest/lnd_channel_funding_fund_max_test.go Outdated Show resolved Hide resolved

ellemouton requested review from sputn1ck and removed request for Crypt-iQ and ellemouton March 11, 2024 13:42

guggero removed their request for review March 13, 2024 16:30

bennyhodl mentioned this pull request Mar 27, 2024

Specify OutPoint in SendCoinsRequest Synota-io/tonic_lnd#12

Draft

Chinwendu20 force-pushed the sendcoins branch 3 times, most recently from c800df5 to 3e6ab6a Compare March 29, 2024 11:27

Chinwendu20 force-pushed the sendcoins branch 4 times, most recently from 7b3d4ac to 6659a2e Compare May 13, 2024 07:37

hieblmi reviewed May 13, 2024

View reviewed changes

Chinwendu20 mentioned this pull request May 13, 2024

wallet: Return an error if select utxos in txtoOutputs contain duplicates. btcsuite/btcwallet#928

Open

Chinwendu20 force-pushed the sendcoins branch from 6659a2e to 9acdfcc Compare May 14, 2024 07:09

Chinwendu20 requested a review from hieblmi May 14, 2024 07:13

Chinwendu20 force-pushed the sendcoins branch from 9acdfcc to ad55793 Compare May 14, 2024 08:58

Chinwendu20 force-pushed the sendcoins branch 3 times, most recently from 607c321 to 27d8cdf Compare May 22, 2024 12:06

Chinwendu20 added 9 commits May 28, 2024 13:19

mod: bump fn package

dbe9bda

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lnrpc: Add Outpoint field in SendCoinsRequest

e3c91b5

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lntest+lnwallet: Adapt walletctrl, UTXO sendcoins

fb99cfe

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

sweep: Add selectUtxos to CraftSweepAllTx args

051c841

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

cmd: Export and move utxosToOutpoint.

5626f05

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lnd: modify sendcoin rpc impl for select utxos

05933c1

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

lncli: Add outpoints flag to sendcoins command

e7b486f

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

itest: Update itest with sendcoins with utxo itest

b4c11ae

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

docs: Added release note.

01f216d

Signed-off-by: Ononiwu Maureen <59079323+Chinwendu20@users.noreply.github.com>

Chinwendu20 force-pushed the sendcoins branch from 27d8cdf to 01f216d Compare May 28, 2024 13:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplify coin selection for sendcoins #8516

Simplify coin selection for sendcoins #8516

Chinwendu20 commented Mar 4, 2024 •

edited

coderabbitai bot commented Mar 4, 2024 •

edited

Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

Chinwendu20 commented Mar 4, 2024

Chinwendu20 commented Mar 4, 2024 •

edited

Chinwendu20 commented Mar 5, 2024

bennyhodl commented Mar 12, 2024

bennyhodl commented Mar 26, 2024

Chinwendu20 commented Mar 26, 2024

hieblmi left a comment

hieblmi May 13, 2024

hieblmi May 13, 2024

hieblmi May 13, 2024

Chinwendu20 May 13, 2024

Chinwendu20 commented May 14, 2024

lightninglabs-deploy commented May 21, 2024

hieblmi commented May 24, 2024

Chinwendu20 commented May 24, 2024 •

edited

Chinwendu20 commented May 24, 2024

hieblmi commented May 27, 2024 •

edited

Chinwendu20 commented May 27, 2024 •

edited

Chinwendu20 commented May 28, 2024

Simplify coin selection for sendcoins #8516

Are you sure you want to change the base?

Simplify coin selection for sendcoins #8516

Conversation

Chinwendu20 commented Mar 4, 2024 • edited

Change Description

Steps to Test

Pull Request Checklist

Testing

Code Style and Documentation

coderabbitai bot commented Mar 4, 2024 • edited

Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

Chinwendu20 commented Mar 4, 2024

Chinwendu20 commented Mar 4, 2024 • edited

Chinwendu20 commented Mar 5, 2024

bennyhodl commented Mar 12, 2024

bennyhodl commented Mar 26, 2024

Chinwendu20 commented Mar 26, 2024

hieblmi left a comment

Choose a reason for hiding this comment

hieblmi May 13, 2024

Choose a reason for hiding this comment

hieblmi May 13, 2024

Choose a reason for hiding this comment

hieblmi May 13, 2024

Choose a reason for hiding this comment

Chinwendu20 May 13, 2024

Choose a reason for hiding this comment

Chinwendu20 commented May 14, 2024

lightninglabs-deploy commented May 21, 2024

hieblmi commented May 24, 2024

Chinwendu20 commented May 24, 2024 • edited

Chinwendu20 commented May 24, 2024

hieblmi commented May 27, 2024 • edited

Chinwendu20 commented May 27, 2024 • edited

Chinwendu20 commented May 28, 2024

Chinwendu20 commented Mar 4, 2024 •

edited

coderabbitai bot commented Mar 4, 2024 •

edited

CodeRabbit Configration File (`.coderabbit.yaml`)

Chinwendu20 commented Mar 4, 2024 •

edited

Chinwendu20 commented May 24, 2024 •

edited

hieblmi commented May 27, 2024 •

edited

Chinwendu20 commented May 27, 2024 •

edited