feat: add button to auto-balance LN channels

[uwla]

Solves #831.

NOTE: work in progress, implementation not ready yet.

Done

• Add new action in network store to autobalance channels.
• Add button component that dispatch that action

To do

• Make it work
• Add tests

Help needed here.

Trying to auto-balance the channel I got the error Insufficient balance, but this should not happen since the source has more than enough funds to pay the target and that can be confirmed by logging the amounts in the console.

I decided to open the PR anyway, so I can get help with the bug.

[uwla]

I need some help here. After fixing the mentioned bug, I can add tests and finish the remaining work in the PR.

[jamaljsr]

Trying to auto-balance the channel I got the error Insufficient balance, but this should not happen since the source has more than enough funds to pay the target and that can be confirmed by logging the amounts in the console.

I tested this PR and it's working fine for me. I'm not sure why you're getting that error on your end. Can you share more of the LND logs from one of the nodes?

Since it works for me, I left some feedback on the code.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 17.77778% with 37 lines in your changes are missing coverage. Please review.

Project coverage is 99.16%. Comparing base (4b2c287) to head (01e3f8c).
Report is 9 commits behind head on master.

❗ Current head 01e3f8c differs from pull request most recent head 4cd78dd. Consider uploading reports for the commit 4cd78dd to get more accurate results

Files	Patch %	Lines
src/store/models/network.ts	0.00%	27 Missing ⚠️
src/utils/network.ts	12.50%	7 Missing ⚠️
src/components/designer/AutoBalanceButton.tsx	66.66%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##            master     #859      +/-   ##
===========================================
- Coverage   100.00%   99.16%   -0.84%     
===========================================
  Files          140      140              
  Lines         4582     4447     -135     
  Branches       851      862      +11     
===========================================
- Hits          4582     4410     -172     
- Misses           0       37      +37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[uwla]

@jamaljsr I addressed nearly all issues except the dropdown menu. I can implement tests for the added functionality, but I stil get the following error:

lndService.ts:160 Uncaught (in promise) Error: insufficient_balance
    at LndService.payInvoice (lndService.ts:160)
    at async Object.fn (lightning.ts:249)
    at async Promise.all (:3000/index 0)
    at async Object.fn (network.ts:972)
VM7596:2 Uncaught ReferenceError: process is not defined
    at Object.4043 (<anonymous>:2:13168)
    at r (<anonymous>:2:306599)
    at Object.8048 (<anonymous>:2:9496)
    at r (<anonymous>:2:306599)
    at Object.8641 (<anonymous>:2:1379)
    at r (<anonymous>:2:306599)
    at <anonymous>:2:315627
    at <anonymous>:2:324225
    at <anonymous>:2:324229
    at HTMLIFrameElement.e.onload (index.js:1)

You said it worked on your side, so I ask you to test again. I'll try to figure out why it is not working on my machine.

Actually, I'm getting some buggy behavior unrelated to the PR: sometimes I can't click on anything, which I suppose not to be a polar bug but a bug with electron due to my window manager.

[uwla]

I'll be back to it in a few days.

[jamaljsr]

Hey @uwla are you planning to continue to work on this PR?

I took a look at your latest changes and noticed what is causing the insufficient_balance error. You have the target and source swapped in the autoBalanceChannels function. The target should be the node creating the invoice and the source should pay it.

        promisesToAwait.push(
          createInvoice({ node: target, amount }).then(invoice =>
            payInvoice({ node: source, amount, invoice }),
          ),
        );

[uwla]

Hey @uwla are you planning to continue to work on this PR?

I took a look at your latest changes and noticed what is causing the insufficient_balance error. You have the target and source swapped in the autoBalanceChannels function. The target should be the node creating the invoice and the source should pay it.

        promisesToAwait.push(
          createInvoice({ node: target, amount }).then(invoice =>
            payInvoice({ node: source, amount, invoice }),
          ),
        );

Thanks @jamaljsr ! I am back to the PR, working on it today

[uwla]

I got some problems:

• LND can pay to LND
• LND can pay to CoreLightning
• LND cannot pay to Eclair neither Eclair can pay to LND:

Uncaught (in promise) Error: no_route
    at LndService.payInvoice (lndService.ts:160)
    at async Object.fn (lightning.ts:249)
    at async Promise.all (:3000/index 0)
    at async Object.fn (network.ts:971)
Uncaught ReferenceError: process is not defined
    at Object.4043 (<anonymous>:2:13168)
    at r (<anonymous>:2:306599)
    at Object.8048 (<anonymous>:2:9496)
    at r (<anonymous>:2:306599)
    at Object.8641 (<anonymous>:2:1379)
    at r (<anonymous>:2:306599)
    at <anonymous>:2:315627
    at <anonymous>:2:324225
    at <anonymous>:2:324229
    at HTMLIFrameElement.e.onload (index.js:1)

• CoreLightning cannot pay to CoreLightning:

Uncaught (in promise) Error: lightningd -32602: msatoshi parameter unnecessary
    at request (clightningApi.ts:38)
    at processTicksAndRejections (internal/process/task_queues.js:93)
    at async CLightningService.payInvoice (clightningService.ts:173)
    at async Object.fn (lightning.ts:249)
    at async Promise.all (:3000/index 0)
    at async Object.fn (network.ts:971)

After playing a bit, I found out I am also getting the same behavior when generating and paying invoices manually with the mentioned implementations: Create invoice -> Copy and Close -> Pay invoices.

The only which works pretty well is LND to LND.

[jamaljsr]

Thanks for the details on the issues your facing. I think I've found solutions to your outstanding problems.

LND cannot pay to Eclair neither Eclair can pay to LND:

I had to dive pretty deep into the weeds to figure this one out. I was able to reproduce the behavior on my end as well. The problem is essentially a configuration issue on the Eclair side. By default, it won't let you make a single payment that is more than 45% of the channel capacity. Check the reference.conf for a more detailed explanation. It would be nice if Eclair logged a clearer error message, but it doesn't. When I changed this to 100%, then Eclair began behaving correctly.

  --channel.max-htlc-value-in-flight-percent=100

Copy this config value into the default Eclair startup flags in constants.ts.

• CoreLightning cannot pay to CoreLightning:

Uncaught (in promise) Error: lightningd -32602: msatoshi parameter unnecessary
    at request (clightningApi.ts:38)
    at processTicksAndRejections (internal/process/task_queues.js:93)
    at async CLightningService.payInvoice (clightningService.ts:173)
    at async Object.fn (lightning.ts:249)
    at async Promise.all (:3000/index 0)
    at async Object.fn (network.ts:971)

This error is actually saying that you should not provide the amount parameter to payInvoice since the invoice was created with an amount specified. The amount field should only be provided to CLN when the invoice contains a zero amount. The other implementations are more forgiving about this.

The API Docs state: "The pay RPC command attempts to find a route to the given destination, and send the funds it asks for. If the bolt11 does not contain an amount, amount_msat is required, otherwise if it is specified it must be null."

I also saw this error in my testing and remove the amount when calling payInvoice fixed it for me.

[jamaljsr]

After testing this a bit more, I feel like immediately balancing the channels when the button is clicked can be a bit unexpected. It would be very cool if it instead displayed a modal with the details of the payments that will be made on each channel. Then when the Submit button in the modal is clicked, the channels would get balanced. What do you think about this idea?

[uwla]

@jamaljsr I think it is a good idea.

So, we show a modal and for each channel:

• left side shows the opening node, with input field to choose amount
• followed by a slider
• right side shows the other node, with input field ot choose amount

After all channels, we add a button on the bottom "auto balance all channels", which will autobalance the preview values.

Finally, at the very bottom, a CONFIRM button an a CANCEL button. If user confirms, each channel will be updated in batch if needed. Otherwise the modal is hidden and its state reset.

What about it now?

[uwla]

I can create the modal and add some tests for the coverage not to drop.

However, I have been facing an weird issue while developing polar locally: after launching the live Electron app, a lot of times I can not click on buttons, and the cursor icon does not alter (it is the same as normal, instead of a click pointer). This has been the most headache thing, because I have to open the devtools and do something like document.querySelector("myquery button").click() to trigger the click and see the effects of my code

[jamaljsr]

Hey, I really like the idea of the sliders. That would be very cool. I'm not sure about the input fields because that can really clutter up the limited space in the small modal. I think it might be easier to implement and more visually appealing to just display the node name and balance on each side with the slider in between.

However, I have been facing an weird issue while developing polar locally: after launching the live Electron app, a lot of times I can not click on buttons, and the cursor icon does not alter (it is the same as normal, instead of a click pointer). This has been the most headache thing, because I have to open the devtools and do something like document.querySelector("myquery button").click() to trigger the click and see the effects of my code

I have also noticed this happening on my machine whenever I save my changes and the UI hot reloads. I'm not sure where this is coming from, but my hunch is it's related to webpack. There is an invisible iframe being injected on the page which covers the whole screen so you can't click on anything.

I have been working around this using two methods depending on whether I want to lose my current place in the UI or not. Note: I'm on a Mac so the keyboard shortcuts may be different if you're on another OS.

1. Hit CMD + SHIFT + C then click anywhere in the UI. This will select the iframe element in the DevTools Elements tab. Then I right-click on the <iframe> DOM element then delete it. Now Im able to interact with the page normally.
2. Just hit CMD + R. This will reload the app and most times the iframe will be gone.

I plan on investigating where this iframe is coming from and fixing it over the next few days. It's been pretty annoying since I've also been working on some updates lately.

[jamaljsr]

After writing the above response, I did a quick google search and found the cause of the issue. It is the react-error-overlay package injecting the iframe. I just needed to force yarn to use v6.0.9 and that resolves the problem. I just pushed 4df612c to master, so if you rebase this branch, you won't need to use the workarounds I suggested. Please let me know if this is still an issue for you.

[uwla]

@jamaljsr I have sync my fork and now the bug is fixed.

I'm working on the dropdown menu and the modal.
It will be awesome when finished!

[uwla]

Hey @jamaljsr, when creating a 250,000 sats channels, which is the default, I noticed if the implementation is LND, then the source starts with 246530 sats and the target 0 sats. Where are the missing sats?

[uwla]

I have not finished the PR, but I pushed it anyway so you can see how it is going.

I added the dropdown and the modal.

I still need to: implement business logic, add tests, code styling refactoring.

[jamaljsr]

Thanks for the update. I will check it out over the next few days.

[uwla]

I'll add business logic today. The laast PR added UI, but it did not implement any logic

[jamaljsr]

Hey @jamaljsr, when creating a 250,000 sats channels, which is the default, I noticed if the implementation is LND, then the source starts with 246530 sats and the target 0 sats. Where are the missing sats?

Sorry, I forgot to answer this question. The balance is decreased by the fee for the commitment transaction and the amount allocated to the anchor output, since Anchor channels are created by default in LND. You can use lncli listchannels to get the commitment fee (commit_fee). For anchor outputs, the amount is always 330 sats.

For example, here's the output for one 250K sat channel I just created.

lnd@alice:/$ lncli listchannels
{
    "channels": [
        {
            "active": true,
            "remote_pubkey": "0347925fd31b4a2c6661fc0cc5e60fea9e549d5f2d94587048b4929aa6d8a2baa7",
            "channel_point": "03a75ce3208bf7d3cfc386d118638e5d0fe47c28caaa063220cda80c12dd6ec2:0",
            "chan_id": "131941395398656",
            "capacity": "250000",
            "local_balance": "246530",
            "remote_balance": "0",
            "commit_fee": "3140",
            "commit_weight": "772",
            "fee_per_kw": "2500",
            "unsettled_balance": "0",
            "total_satoshis_sent": "0",
            "total_satoshis_received": "0",
            "num_updates": "0",
            "pending_htlcs": [],
            "csv_delay": 144,
            "private": false,
            "initiator": true,
            "chan_status_flags": "ChanStatusDefault",
            "local_chan_reserve_sat": "2500",
            "remote_chan_reserve_sat": "2500",
            "static_remote_key": false,
            "commitment_type": "ANCHORS",
            "lifetime": "9",
            "uptime": "9",
            "close_address": "",
            "push_amount_sat": "0",
            "thaw_height": 0,
            "local_constraints": {
                "csv_delay": 144,
                "chan_reserve_sat": "2500",
                "dust_limit_sat": "354",
                "max_pending_amt_msat": "247500000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 483
            },
            "remote_constraints": {
                "csv_delay": 144,
                "chan_reserve_sat": "2500",
                "dust_limit_sat": "354",
                "max_pending_amt_msat": "247500000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 483
            },
            "alias_scids": [],
            "zero_conf": false,
            "zero_conf_confirmed_scid": "0",
            "peer_alias": "bob",
            "peer_scid_alias": "0",
            "memo": ""
        }
    ]
}

The balance breakdown is as follows:

    250,000   <- Channel capacity
-     3,140   <- Commitment transaction fee
-       330   <- Anchor output
-----------
    246,530   <- Local balance

[uwla]

Thanks @jamaljsr for the updates.

My last push had some UI improvements. I will implement business logic tomorrow, by adding a new store action. Things are working pretty well now that I can click buttons and test things with hot-reload, after fixing the react overlay bug.

[uwla]

With the last push, it is WORKING!

At least with LND nodes. It stills does not work well with Clightning and Eclair due to the mentioned bugs, which we were discussing previously.

Now, what remains:

1. solve the issues with clightning and eclair
2. add automated tests
3. minor UI improvents: nicer buttons and autoclose the modal afterwards
4. refactoring (I just want a working prototype, so I may have added functions and interfaces in the wrong places)

@jamaljsr I need your help with (1), since you know what is causing it. I also need your opinion with (4), because I am not familiar with Polar's preferences for code style, where the added functions and interfaces should be placed.

[jamaljsr]

@jamaljsr I need your help with (1), since you know what is causing it. I also need your opinion with (4), because I am not familiar with Polar's preferences for code style, where the added functions and interfaces should be placed.

Did you apply the fixes I mentioned in this prior comment? This resolved the Core Lightning and Eclair issues for me.

4. refactoring (I just want a working prototype, so I may have added functions and interfaces in the wrong places)

I will review your latest changes by tomorrow for sure.