Deniability - a tool to automatically improve coin ownership privacy

[denavila]

This new feature is an implementation of the ideas in Paul Sztorc's blog post "Deniability - Unilateral Transaction Meta-Privacy"(https://www.truthcoin.info/blog/deniability/).

In short, the idea is to periodically split coins and send them to yourself, making it look like common "spend" transactions, such that blockchain ownership analysis becomes more difficult, and thus improving the user's privacy.

This is the GUI portion of the PR (bitcoin-core/gui). The core functionality PR ( bitcoin/bitcoin#27792 ) is in the main repo (bitcoin/bitcoin).

This PR implements an additional "Deniability" wallet view. The majority of the GUI code is in a new deniabilitydialog.cpp/h source files containing a new DeniabilityDialog class, hooked up via the WalletView class. 

On startup and on notable events (new blocks, new transactions, etc), we evaluate the privacy of all coins in the wallet, and we build a "deniabilization" candidate list. UTXOs that share the same destination address are grouped together into a single candidate (see DeniabilityDialog::updateCoins and DeniabilityDialog::updateCoinTable).

We inspect the blockchain data to find out if we have performed "deniabilization" transactions already, and we count how many "cycles" (self-sends) have been performed for each coin (see DeniabilityDialog::calculateDeniabilizationStats). Since we infer the state entirely from the blockchain data, even if the wallet is restored from a backup, the state would not be lost. This also means that if the user has performed manual self-sends that have improved the ownership privacy, they will be counted too.

The user has a couple of controls for the "deniabillization" process - a Frequency selector and a Budget spinner, which respectively determine how often to perform a cycle (see below) and how much total BTC to spend on transaction fees.

The user can initiate the "deniabillization" process by pressing the Start button (DeniabilityDialog::startDeniabilization).
The first time the button is pressed, we offer an information / confirmation box to explain to the user how Deniability works.

The process periodically perform a "deniabilization" cycle (DeniabilityDialog::deniabilizeProc). In each such cycle we do the following:
A coin is selected form the candidate list. The more a coin is "deniabilized", the less likely it is to be selected. Smaller coins are also less likely to be selected. If a coin is selected, we prepare and broadcast a transaction, which splits the coin into a pair of new wallet addresses (DeniabilityDialog::deniabilizeCoin). 

If Bitcoin Core is left running continuously, the cycles would be performed at the selected frequency (with some randomization). If Bitcoin Core is shutdown, the "deniabilization" process will resume at the next restart, and if more time has elapsed than the selected frequency, it will perform a single cycle. We deliberately don't "catch up" all missed cycles, since that would expose the process to blockchain analysis. The state is saved and restored via QSettings (DeniabilityDialog::loadSettings and DeniabilityDialog::saveSettings).

We monitor each broadcasted transaction and we automatically attempt a fee bump if the transaction is still in the memory pool since the previous cycle (DeniabilityDialog::bumpDeniabilizationTx). We don't issue any other deniabilization transactions until the previous transaction is confirmed (or abandoned/dropped).

The process ends when the budget is exhausted or if there's no candidates left. The user can also stop the process manually by pressing a Stop button (DeniabilityDialog::stopDeniabilization).

External signers are supported in a "best effort" way - since the user needs to manually sign, we postpone the deniabilization processing till the external signer is connected and use some additional UI to get the user's attention to sign (see the codepath predicated on hasExternalSigner). This is not ideal, so I'm looking for ideas if we can improve this in some way.

Watch-only wallets are partially supported, where we display the candidate list, but we don't allow any processing (since we don't have the private keys to issue transactions).

I've tested all this functionality on regtest, testnet, signet and mainnet. I've also added some unit tests (under WalletTests) to exercise the main functions.

This is my first change and PR for Bitcoin Core, and I did my best to validate everything against the guidelines and documentation and to follow the patterns in the existing code, but I'm sure there may be things I've missed, so I'm looking forward to your feedback.
In particular one thing I'm not particularly happy with - the save/restore of state via QSettings makes me a bit nervous as we store wallet specific data keyed on wallet name, however wallets can be erased and new ones created with the same name, so this is not ideal. I made an effort to validate all data on load, but it would be great if we could improve this somehow - either by storing the settings based on some wallet identity signature, or by storing the state in the wallet database (however that doesn't seem accessible via the interfaces::Wallet API). Please let me know your thoughts and suggestions.

Thank you.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	katesalazar

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #815 (Bugfix on TransactionsView - Disable if privacy mode is set during wallet selection by pablomartin4btc)
• #bitcoin/bitcoin/29790 ([DO NOT MERGE] cmake: Migrate CI scripts to CMake-based build system -- WIP by hebasto)
• #bitcoin/bitcoin/29564 (wallet: clarify FundTransaction signature by S3RK)
• #bitcoin/bitcoin/27865 (wallet: Track no-longer-spendable TXOs separately by achow101)
• #bitcoin/bitcoin/27286 (wallet: Keep track of the wallet's own transaction outputs in memory by achow101)
• #bitcoin/bitcoin/24963 (RPC/Wallet: Convert walletprocesspsbt to use options parameter by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[hebasto]

This PR is an implementation of the ideas in Paul Sztorc's blog post "Deniability - Unilateral Transaction Meta-Privacy"(truthcoin.info/blog/deniability).

Has that idea been discussed somewhere already?

[denavila]

Has that idea been discussed somewhere already?

The idea was published quite a while ago, but I learned about it from some Bitcoiner friends just recently.
We looked around to see if there were existing discussions or implementations, but we didn't find anything on bitcoin-dev or the repo here.
There's an old implementation in a Bitcoin fork (Drivechain) which I used as a starting point, but ended up with a pretty different approach and implementation.
We were thinking of perhaps writing down the spec into a Process BIP, if this would make it easier to review on a conceptual level?

[VladdyC]

This is really cool and I'd love to have it as an option in Bitcoin Core. However, most CoinJoin implementations are more advanced in this regard – for example, WabiSabi performs both divisions and consolidations. And it makes me wonder to which extent it's possible to decentralize coordination to the point where clients randomly take turns in doing it.

Anyway, this is just my curiosity/wish list. I believe that anything is better than having no privacy options at all, so I'd be happy with some more basic deniability too :)

[achow101]

It seems like a feature like this would be better to integrate into the wallet directly rather than something that is done through and managed by the GUI.

[denavila]

This is really cool and I'd love to have it as an option in Bitcoin Core. However, most CoinJoin implementations are more advanced in this regard – for example, WabiSabi performs both divisions and consolidations. And it makes me wonder to which extent it's possible to decentralize coordination to the point where clients randomly take turns in doing it.

Yes, CoinJoin is definitely more powerful, however it has some drawbacks.
In particular, if it's possible to detect that an UTXO has come from a coinjoin, it may be considered "tainted" by CEXs and they could refuse processing it (eg demand source of funds etc).

That's the "meta-privacy" that Paul talks about in his blog post. The idea that the knowledge of a secret has to also be kept a secret.

Here's another one of Paul's blog posts where he talks about drawbacks of other privacy methods: https://www.truthcoin.info/blog/expensive-privacy/

At any rate, I'm definitely open to suggestions or ideas we can borrow from CoinJoin.
In particular if there's a way we can re-merge UTXOs in a privacy-preserving way. Or at bare minimum, use "deniability-aware" logic in the automatic coin selection during user-initiated spends to minimize the privacy reduction.

Anyway, this is just my curiosity/wish list. I believe that anything is better than having no privacy options at all, so I'd be happy with some more basic deniability too :)

Thank you. I believe so too.

[denavila]

It seems like a feature like this would be better to integrate into the wallet directly rather than something that is done through and managed by the GUI.

Do you mean moving some of the core logic inside the interfaces::Wallet API, or the CWallet implementation?

Yes, certainly, that could have benefits. I had to workaround certain things that weren't in the Wallet API (namely address reservation, bump fee handling etc). So that would clean up the wallet side for sure.

Since this was my first Bitcoin Core change I decided to be more conservative and avoid API changes.
Of course, if this PR is accepted, I'd be happy to move functionality into the API.

[achow101]

Do you mean moving some of the core logic inside the interfaces::Wallet API, or the CWallet implementation?

Inside of CWallet.

Since this was my first Bitcoin Core change I decided to be more conservative and avoid API changes.
Of course, if this PR is accepted, I'd be happy to move functionality into the API.

The API isn't public, so changing it isn't a big concern. Really the only consumer of it is the GUI.

Having it inside of CWallet will probably make this feature more likely to be accepted. It will be easier to write tests (adding an RPC allows for functional tests, and implementing in CWallet allows unit tests to better access the data) and it can be split up into at least 2 chunks - the CWallet and RPC only implementation, and then the GUI and interface changes. This makes it more likely to be accepted.

[denavila]

The API isn't public, so changing it isn't a big concern. Really the only consumer of it is the GUI.

Having it inside of CWallet will probably make this feature more likely to be accepted. It will be easier to write tests (adding an RPC allows for functional tests, and implementing in CWallet allows unit tests to better access the data) and it can be split up into at least 2 chunks - the CWallet and RPC only implementation, and then the GUI and interface changes. This makes it more likely to be accepted.

Ah, I see. I had assumed the opposite - that not touching the API would be easier to accept.
Ok, I can look into refactoring the PR and moving the core functionality into CWallet.

I'd need to open a PR against the main repo in that case, right?
Should I keep this PR and leave the GUI bits here, and open another PR in the main repo with just the CWallet changes?
Or is it better to close this PR and open a new one in the main repo with all the changes?

[achow101]

I'd need to open a PR against the main repo in that case, right?

Yes

Should I keep this PR and leave the GUI bits here, and open another PR in the main repo with just the CWallet changes?
Or is it better to close this PR and open a new one in the main repo with all the changes?

You can leave this open and rebase it onto the main repo PR, just mention it in the description.

[denavila]

You can leave this open and rebase it onto the main repo PR, just mention it in the description.

All right, I'll start working on this refactor.
Btw, do we have some docs on how to setup git locally so I can have dependent changes across the "bitcoin" and the "gui" repos? So far I was moving changes via patches but that doesn't seem too convenient.

[denavila]

I refactored the code and updated the PR. The new code is split between two branches:

1. deniability-api - new API and implementation of core "deniabilization" functions. I'll open a PR in the main repo once I add RPC and unit tests.
2. deniability - the GUI portion of the Deniability dialog which uses the new deniability-api functions.

[DrahtBot]

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin-core/gui/runs/20890537182}

[1440000bytes]

I tested it and this is the first transaction after starting hourly deniability: https://mempool.space/signet/tx/9acd8fd572d4c245c7e09c454cc721caa2ce4dc91d970400ba228edc81e15d16

Some feedback based on testing:

1. 'Budget' should be called 'Fee Budget'.
2. There should be an option to select "random" frequency.
3. There should be an option to select minimum amount for resulting UTXOs so that user doesn't end up with dust outputs that need to be used together.
4. Post deniability UTXOs have to be carefully spent but there is nothing to differentiate them in the labels and all are labelled as "deniability". Users will need to do manual analysis before spending which is not good UX.
5. I don't understand why the next cycle starts after 1 hr 34 minutes when frequency is 1 hour: https://i.imgur.com/fxE0AvG.png

[denavila]

@1440000bytes,
Thank you so much for testing the PR and for the feedback. That's very helpful.

(1) Yes, that's a good idea, I'll rename it.
(2) and (5) are related. The selectable frequencies are actually also randomized, within the selected frequency range - for example hourly is random from 1 to 2 hours, daily is random from 1 to 2 days etc (see line 934 in deniabilitydialog.cpp). Perhaps we need to rename the frequency UI selection to something that better describes this?
(3) Great idea as well - especially since we're getting into long stretches of overloaded mempool with fees sky rocketing. I'm actually even more concerned that any number the user picks, may not be good enough long term, as smaller UTXOs may effectively become dust if fees rise and stay elevated (and there's a good chance that's going to happen). I'm not sure we can solve that just on the transaction end though ... A minimum UTXO amount does sound like a good option regardless, so I'll implement that.
(4) Indeed - I'll have to think how to solve this. Perhaps we need to add some "deniability" awareness to the regular UTXO selection logic. I'm not entirely clear what that logic would look like though. Let me know if you have ideas - eg, what kind of rules for UTXO selection should we add etc.

I'm a bit busy with my main job at the moment, but when I get a chance I'll work on this as time allows.
Thanks!

[DrahtBot]

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin-core/gui/runs/22114359812}