Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update BIP 21 with information about more modern usage of it #1555

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

TheBlueMatt
Copy link
Contributor

As Bitcoin has grown, the introduction of new address formats describing new forms of payment instructions has become increasingly fraught with compatibility issues. Not only does there exist traditional on-chain addresses, but some recipients wish to receive Lightning (when the sender supports it) or newer formats such as Silent Payments.

This has led to increasing use of the BIP 21 query parameters to encode further optional payment instructions.

Looking forward, as new payment instructions get adopted, it makes much more sense to include them in query parameters rather than replace the existing address field, ensuring compatibility with senders and recipients who may or may not be upgraded to support all the latest payment instructions.

This updates BIP 21 to suggest that future address formats do this.

Further, it updates BIP 21 to allow an empty bitcoin address in cases where new payment instructions have moved to becoming mandatory. This isn't a backwards-incompatible change any more than switching to a new address format is, so doesn't impact existing BIP 21 implementations in a new way, however provides a nice conclusion to the query-parameter-based upgrade path - once a form of payment instructions has broad adoption, senders can simply drop the existing address field, keeping their existing query parameter encoding, rather than replace the existing address field. It also addresses the question of what to do if a wallet no longer wishes to receive some legacy on-chain address, but has multiple payment instruction formats that they wish to include - deciding which one to place in the address field would be a difficult task.

As Bitcoin has grown, the introduction of new address formats
describing new forms of payment instructions has become
increasingly fraught with compatibility issues. Not only does there
exist traditional on-chain addresses, but some recipients wish to
receive Lightning (when the sender supports it) or newer formats
such as Silent Payments.

This has led to increasing use of the BIP 21 query parameters to
encode further optional payment instructions.

Looking forward, as new payment instructions get adopted, it makes
much more sense to include them in query parameters rather than
replace the existing address field, ensuring compatibility with
senders and recipients who may or may not be upgraded to support
all the latest payment instructions.

This updates BIP 21 to suggest that future address formats do this.

Further, it updates BIP 21 to allow an empty bitcoin address in
cases where new payment instructions have moved to becoming
mandatory. This isn't a backwards-incompatible change any more than
switching to a new address format is, so doesn't impact existing
BIP 21 implementations in a new way, however provides a nice
conclusion to the query-parameter-based upgrade path - once a form
of payment instructions has broad adoption, senders can simply drop
the existing address field, keeping their existing query parameter
encoding, rather than replace the existing address field. It also
addresses the question of what to do if a wallet no longer wishes
to receive some legacy on-chain address, but has multiple payment
instruction formats that they wish to include - deciding which one
to place in the address field would be a difficult task.
@josibake
Copy link
Member

josibake commented Mar 4, 2024

(Background discussion for context: https://delvingbitcoin.org/t/revisiting-bip21/630)

Thanks for starting this! Conceptually, I agree with the updates but I think we can get a bigger win by advising the use of HRPs directly instead of key-value pairs. The benefits of this approach are:

  • Better taproot support: using HRPs directly would allow us to construct backwards compatible taproot URIs of the form bitcoin:bc1q...?bc1p...=o / bitcoin:bc1q...?bc1p...
  • Support for future payment instructions: any new payment protocol that encodes their payment instructions using bech32m can be included directly, e.g. bitcoin:bc1q...?newprotocol1<bech32m encoded data>=o
  • Existing unified QR codes can be made smaller: following an upgrade period to allow clients to update, we would be able to create URIs bitcoin:bc1q...?lnbc1...=o (instead of ?lightning=lnbc1...) and fully static URIs bitcoin:sp1q...?lno1...=o

For senders, this simplifies implementing support for new address types in that clients can implement support for a generic BIP21 URI using HRPs as keys. As the client supports new bech32m encoded addresses, they are supported automatically without any additional changes.

Clients would still need to support new payment instructions that instead decided to use a query parameter, but I would expect most (if not all) clients to prefer bech32m encodings now that they get BIP21 support for free.

I wrote a rough draft here, feel free use / modify as needed if you find it useful: josibake@07339bd

@TheBlueMatt
Copy link
Contributor Author

Better taproot support: using HRPs directly would allow us to construct backwards compatible taproot URIs of the form bitcoin:bc1q...?bc1p...=o / bitcoin:bc1q...?bc1p...

I think this ship has sailed, but K/V-vs-no-K has no impact on this. We could do bitcoin:bc1q...?taproot=bc1p. or whatever just fine. Ultimately its probably too late to update how any taproot anything appears in QR codes/URIs.

Support for future payment instructions

This is similarly untrue, the only difference is it reduces the characters used for future instructions, but whether it supports future instructions or not, both do.

Existing unified QR codes can be made smaller: following an upgrade period to allow clients to update, we would be able to create URIs bitcoin:bc1q...?lnbc1...=o (instead of ?lightning=lnbc1...) and fully static URIs bitcoin:sp1q...?lno1...=o

Indeed, we can save a few characters here or there. I think the ship has similarly sailed for BOLT 11, but of course we can do something different for BOLT 12.

Ultimately I think the only difference between the two proposals are:

  • Skipping the key has slightly less bytes in the QR code, which helps very slightly on the margin.
  • Skipping the key means parsing is a bit trickier if/when we have some new payment instructions that don't use bech32m - do clients need to check the bech32m checksum for unknown payment instruction types? What do they do if its wrong? What happens when someone (without thinking) defines some payment instructions that match a bech32m HRP spuriously (but I guess probably the checksum would be wrong?). These should all be written out and considered if we want to go this path.

I think the right approach here is the simpler one, but there's not a really strong reason to prefer either over the other, honestly.

@josibake
Copy link
Member

josibake commented Mar 7, 2024

This is similarly untrue, the only difference is it reduces the characters used for future instructions, but whether it supports future instructions or not, both do.

No, they are not the same. This is especially relevant if wallets are using a BIP21 library: my wallet supports new address type abc1xxxx, which (according to your proposal) also gets a abc key defined (i.e. abc=abc1xxx. My wallet can parse the address but since my BIP21 library I am using hasn't added support for the new key, I am unable to parse these URIs. With my proposal of allowing bech32m encoded addresses to be used without a key, everything Just Works.

Skipping the key means parsing is a bit trickier if/when we have some new payment instructions that don't use bech32m - do clients need to check the bech32m checksum for unknown payment instruction types? What do they do if its wrong? What happens when someone (without thinking) defines some payment instructions that match a bech32m HRP spuriously (but I guess probably the checksum would be wrong?). These should all be written out and considered if we want to go this path.

I'm not really sure what you're getting at here? My proposal is that any new payment addresses must use bech32m if they want to be used without a key, otherwise they must define a key. Everything you just mentioned was predicated on the assumption "what if they don't use bech32m and don't define a key," which means they wouldn't be following the spec.

@TheBlueMatt
Copy link
Contributor Author

No, they are not the same. This is especially relevant if wallets are using a BIP21 library: my wallet supports new address type abc1xxxx, which (according to your proposal) also gets a abc key defined (i.e. abc=abc1xxx. My wallet can parse the address but since my BIP21 library I am using hasn't added support for the new key, I am unable to parse these URIs. With my proposal of allowing bech32m encoded addresses to be used without a key, everything Just Works.

That applies both to a K/V parameter and a non-K/V parameter equally - there's really no difference here. A BIP21 parsing library should pass all parameters that it doesn't know.

I'm not really sure what you're getting at here? My proposal is that any new payment addresses must use bech32m if they want to be used without a key, otherwise they must define a key. Everything you just mentioned was predicated on the assumption "what if they don't use bech32m and don't define a key," which means they wouldn't be following the spec.

Ah, okay, I misunderstood the proposal. I'm not really super excited to bake "future addresses will use bech32m" into the spec in that way, because at some point we're gonna want "bech32n" or some other encoding (which would make sense for stuff that's only in QR codes as you could get the QR a bit denser) and then we'll be back having this same discussion, except now we have to shove everything in K/V pairs because we restricted non-K/V pairs to bech32m-only.

@josibake
Copy link
Member

josibake commented Mar 7, 2024

I'm not really super excited to bake "future addresses will use bech32m" into the spec in that way, because at some point we're gonna want "bech32n"

While certainly not perfect, I think this is better than the alternative of whitelisting a set of addresses that are allowed in a root in this BIP and requiring new formats to specify extension keys. My proposal gives us a way to specify a taproot address in a backwards compatible way, it allows for clients to save space by not needing to redundantly specify hrp=hrp..., leaves open the possibility for implementations to move to use the BOLT11 HRP directly to save space, and provides some future proofing for new address formats insomuch as bech32m continues to be the standard.

@TheBlueMatt
Copy link
Contributor Author

While certainly not perfect, I think this is better than the alternative of whitelisting a set of addresses that are allowed in a root in this BIP and requiring new formats to specify extension keys. My proposal gives us a way to specify a taproot address in a backwards compatible way, it allows for clients to save space by not needing to redundantly specify hrp=hrp..., leaves open the possibility for implementations to move to use the BOLT11 HRP directly to save space, and provides some future proofing for new address formats insomuch as bech32m continues to be the standard.

To be clear, I think we should "whitelist the set of addresses that are allowed in the root" either way. IMO it was a (now-clear) mistake to have taproot at the root rather than in a parameter. Whether we go with K/V or not-K/V we still want to have all future address types in parameters rather than the URI root (and eventually basically phase out the URI root entirely, or at least make it taproot-only).

@josibake
Copy link
Member

josibake commented Mar 8, 2024

To be clear, I think we should "whitelist the set of addresses that are allowed in the root" either way.

Effectively, this is what you get with my proposal:

The bitcoinaddress body MUST be either a legacy base58 address (P2PKH, P2SH), or a bech32(m) encoded address. Future address formats that do not use bech32m encoding MUST instead be placed in query keys. Query keys SHOULD be defined by the respective BIP for the new address format.

The only distinction is newer bech32m address types can also be placed in the root. If you're planning to allow bitcoin:?hrp=hrpxxx...&anotherhrp=anotherhrpxxx&amount=<>, that's exactly the same as bitcoin:hrpxxx...?anotherhrpxxx&amount=<>, just more compact.

@TheBlueMatt
Copy link
Contributor Author

Effectively, this is what you get with my proposal:

This is unrelated to the K/V/no-K/V discussion. We can get it either way.

The only distinction is newer bech32m address types can also be placed in the root.

I don't think we should allow this. It would be nice to only have one place to look for a given address type.

@murchandamus
Copy link
Contributor

I noticed that there is another pending PR that seeks to amend BIP21 #1394. At first glance, it seems like the change suggested there could be incorporated here.

@@ -39,7 +39,7 @@ Elements of the query component may contain characters outside the valid range.

(See also [[#Simpler syntax|a simpler representation of syntax]])

bitcoinurn = "bitcoin:" bitcoinaddress [ "?" bitcoinparams ]
bitcoinurn = "bitcoin:" [ bitcoinaddress ] [ "?" bitcoinparams ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not been following, but did you consider rejecting empty string after colon, and removing unnecessary interrogation character?

Suggested change
bitcoinurn = "bitcoin:" [ bitcoinaddress ] [ "?" bitcoinparams ]
bitcoinurn = "bitcoin:" ( bitcoinaddress [ "?" bitcoinparams ] | bitcoinparams )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants