Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backport: bitcoin#19136, #21063, #21277, #21302, partial #20267 - descriptor wallets part IV #5981

Merged
merged 11 commits into from
Apr 16, 2024

Conversation

knst
Copy link
Collaborator

@knst knst commented Apr 10, 2024

Issue being fixed or feature implemented

https://github.com/dashpay/dash-issues/issues/59

Extra notes

This commit chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test" is not directly connected to descriptor wallets, but added to this PR due to conflicts with 20267

What was done?

It steadily improves support of descriptor wallets in Dash core.

Done backports and related fixes:

Beside backports and related fixes, this PR includes release notes for previous batch of backports for descriptor wallets support #5965

How Has This Been Tested?

Run unit functional tests

Breaking Changes

N/A

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone

laanwj and others added 4 commits April 11, 2024 02:37
…t compiled

Backport notice: changes in feature_notification.py are missing due to bitcoin#18878 is not done yet

49797c3 tests: Disable bdb dump test when no bdb (Andrew Chow)
1194cf9 Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow)
fbaea7b Require legacy wallet for wallet_upgradewallet.py (Andrew Chow)
b1b679e Explicitly mark legacy wallet tests as such (Andrew Chow)
09514e1 Setup wallets for interface_zmq.py (Andrew Chow)
4d03ef9 Use MiniWallet in rpc_net.py (Andrew Chow)
4de2382 Setup wallets for interface_bitcoin_cli.py (Andrew Chow)
7c71c62 Setup wallets with descriptors for feature_notifications (Andrew Chow)
1f1bef8 Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow)
c77975a Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow)
1f20cac Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow)
3641597 tests: Don't make any wallets unless wallet is required (Andrew Chow)
b9b88f5 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow)
6f36242 tests: Set descriptors default based on compilation (Andrew Chow)

Pull request description:

  This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped.

  For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet.

ACKs for top commit:
  laanwj:
    ACK 49797c3
  ryanofsky:
    Code review ACK 49797c3. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is bitcoin#20267 (review)

Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
…is not compiled)

This partially reverts commit da8e563.
Also it adds missing changed from bitcoin#16404
@knst knst changed the title backport: bitcoin#19136, #20267, #21063, #21277, #21302 - descriptor wallets IV backport: bitcoin#19136, #20267, #21063, #21277, #21302 - descriptor wallets part IV Apr 10, 2024
@knst knst added this to the 21 milestone Apr 10, 2024
@knst knst force-pushed the bp-descriptors-4 branch 2 times, most recently from 4a4f5a8 to c6322ac Compare April 11, 2024 20:05
@knst knst changed the title backport: bitcoin#19136, #20267, #21063, #21277, #21302 - descriptor wallets part IV backport: bitcoin#19136, #21063, #21277, #21302, partial #20267 - descriptor wallets part IV Apr 11, 2024
@knst knst marked this pull request as ready for review April 11, 2024 20:16
@knst knst added the RPC Some notable changes to RPC params/behaviour/descriptions label Apr 11, 2024
@knst knst marked this pull request as draft April 12, 2024 05:30
knst and others added 7 commits April 12, 2024 12:31
de6b389 tests: Test getaddressinfo parent_desc (Andrew Chow)
e4ac869 rpc: Add parent descriptor to getaddressinfo output (Andrew Chow)
bbe4a36 wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan (Andrew Chow)
9be1437 descriptors: Add ToNormalizedString and tests (Andrew Chow)

Pull request description:

  Adds `parent_desc` field to the `getaddressinfo` RPC to export a public descriptor. Using the given address, `getaddressinfo` will look up which `DescriptorScriptPubKeyMan` can be used to produce that address. It will then return the descriptor for that `DescriptorScriptPubKeyMan` in the `parent_desc` field. The descriptor will be in a normalized form where the xpub at the last hardened step is derived so that the descriptor can be imported to other wallets. Tests are added to check that the correct descriptor is being returned for the wallet's addresses and that these descriptors can be imported and used in other wallets.

  As part of this PR, a `ToNormalizedString` function is added to the descriptor classes. This really only has an effect on `BIP32PubkeyProvider`s that have hardened derivation steps. Tests are added to check that normalized descriptors are returned.

ACKs for top commit:
  Sjors:
    utACK de6b389
  S3RK:
    Tested ACK de6b389
  jonatack:
    Tested ACK de6b389 modulo a few minor comments
  fjahr:
    Code review ACK de6b389
  meshcollider:
    Tested ACK de6b389

Tree-SHA512: a633e4a39f2abbd95afd7488484cfa66fdd2651dac59fe59f2b80a0940a2a4a13acf889c534a6948903d701484a2ba1218e3081feafe0b9a720dccfa9e43ca2b
…or form

a69c3b3 wallet: listdescriptors uses normalized descriptor form (Ivan Metlushko)

Pull request description:

  Rationale: show importable descriptors with `listdescriptors` RPC

  It uses bitcoin#19136 to derive xpub at the last hardened step.

  **Before**:
  ```
  [
      {
        "desc": "wpkh(tpubD6NzVbkrYhZ4YUQRJL49TWw1VR5v3QKUNYaGGMUfJUm19x5ZqQ2hEiPiYbAQvD2nHoPGQGPg3snLPM8sjmYpvx7XQhkmyfk8xhsUwKbXzyh/84'/1'/0'/0/*)#p4cn3erf",
        "timestamp": 1613982591,
        "active": true,
        "internal": false,
        "range": [
          0,
          999
        ],
        "next": 0
      },
      ...
  ]
  ```

  **After**:
  ```
  [
    {
      "desc": "wpkh([d4ade89c/84'/1'/0']tpubDDUEYcVXy6Vh5meHvcXN3sAr4k3fWwLZGpAHbkAHL8EnkDxp4d99CjNhJHfM2fUJicANvAKnCZS6XaVAgwAeKYc1KesGCN5qbQ25qQHrRxM/0/*)#8wq8rcft",
      "timestamp": 1613982591,
      "active": true,
      "internal": false,
      "range": [
        0,
        999
      ],
      "next": 0
    },
    ...
  ]
  ```

ACKs for top commit:
  achow101:
    ACK a69c3b3

Tree-SHA512: 4f92e726cb8245aa0b520729cfd272945f0c66830f0555544e0618883aca516318543fa6ab1862058c64b4e4ed54ad1da953e032f4846eef7454122031c1b005
2e5f7de wallet, rpc: update listdescriptors response format (Ivan Metlushko)

Pull request description:

  Update `listdescriptors` response format according to [RPC interface guidelines](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#rpc-interface-guidelines).

  This is a follow up for bitcoin#20226

  **Before:**
  ```
  Result:
  [                               (json array) Response is an array of descriptor objects
    {                             (json object)
      "desc" : "str",             (string) Descriptor string representation
      "timestamp" : n,            (numeric) The creation time of the descriptor
      "active" : true|false,      (boolean) Activeness flag
      "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
      "range" : [                 (json array, optional) Defined only for ranged descriptors
        n,                        (numeric) Range start inclusive
        n                         (numeric) Range end inclusive
      ],
      "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
    },
    ...
  ]
  ```

  **After:**
  ```
  Result:
  {                                 (json object)
    "wallet_name" : "str",          (string) Name of wallet this operation was performed on
    "descriptors" : [               (json array) Array of descriptor objects
      {                             (json object)
        "desc" : "str",             (string) Descriptor string representation
        "timestamp" : n,            (numeric) The creation time of the descriptor
        "active" : true|false,      (boolean) Activeness flag
        "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
        "range" : [                 (json array, optional) Defined only for ranged descriptors
          n,                        (numeric) Range start inclusive
          n                         (numeric) Range end inclusive
        ],
        "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
      },
      ...
    ]
  }
  ```

ACKs for top commit:
  achow101:
    re-ACK 2e5f7de
  meshcollider:
    utACK 2e5f7de
  jonatack:
    re-ACK 2e5f7de

Tree-SHA512: 49bf73e46e2a61003ce594a4bfc506eb9592ccb799c2909c43a1a527490a4b4009f78dc09f3d47b4e945d3d7bb3cd2632cf48c5ace5feed5066158cc010dddc1
…lets

5039e0e test: HelpExampleCliNamed and HelpExampleRpcNamed (Ivan Metlushko)
591735e rpc: Add HelpExampleCliNamed and use it for `createwallet` doc (Wladimir J. van der Laan)
5d5a90e rpc: Add HelpExampleRpcNamed (Ivan Metlushko)

Pull request description:

  Rationale: make descriptor wallets more visible and just a bit easier to setup

  `bitcoin-cli help createwallet`

  **Before**:
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

  **After**
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  > bitcoin-cli createwallet "descriptors" false false "" true true true
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["descriptors", false, false, "", true, true, true]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 5039e0e

Tree-SHA512: d37210e6ce639addee881377092d8f6fb2a537a60a259c561899e24cf68a0254d7ff45a213573c938f626677e46770cd21113aae5974f26c66b9a2e137699c14
Since bitcoin#20267 changes default wallet in functional tests from legacy
wallets to descriptor wallets, we need to enforce --legacy-wallets for
functional tests that used protx which doesn't work yet for descriptor wallets
@knst knst marked this pull request as ready for review April 12, 2024 20:49
Copy link

@UdjinM6 UdjinM6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

Copy link
Member

@PastaPastaPasta PastaPastaPasta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK for merging via merge commit

@PastaPastaPasta PastaPastaPasta merged commit 544d333 into dashpay:develop Apr 16, 2024
9 of 12 checks passed
knst added a commit to knst/dash that referenced this pull request May 6, 2024
Enables for rpc_quorum.py, feature_notifications.py

see dashpay#5981, it partial revert of b20f812
knst added a commit to knst/dash that referenced this pull request May 6, 2024
Enables for rpc_quorum.py, feature_notifications.py

see dashpay#5981, it partial revert of b20f812
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
RPC Some notable changes to RPC params/behaviour/descriptions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants