RFC: Formal description of the RPC API

[maflcko]

Currently (all?) clients manually implement the RPC API, which is problematic, because:

• It leads to accidental implementation bugs, such as unit mistakes (s/vB vs BTC/kvB)
• It is hard to maintain, when the API changes
• It requires effort to implement in a new programming language

So it could make sense to have a formal specification.

Please leave a comment if there are additional aspects to consider here, or if you have ideas for a solution.

One solution for a formal description could be OpenAPI. Generators exist, such as (random example) https://github.com/microsoft/kiota?tab=readme-ov-file#supported-languages

[nflatrea]

Don't we have plethora of well documented guides and specifications on the JSON-RPC api ?

https://developer.bitcoin.org/reference/rpc/
https://wbnns.github.io/bitcoin-dev-docs/reference/rpc/index.html
https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md

https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29

Considering your need for a state-of-the-art framework / wrapper, you can always find a suitable project on Github,
You have, for example, the btcsuite project allowing Go clients building on bitcoin easily. For JSON-RPC wrappers you have https://github.com/btcsuite/btcd/tree/master/rpcclient and the same on C/C++, Rust, Python, Ruby etc..

But it might be interesting to maintain a cross-language suite under the bitcoin core account authority

[maflcko]

Don't we have plethora of well documented guides and specifications on the JSON-RPC api ?

No. I'd highly doubt that there is a better source of the documentation and specification than Bitcoin Core itself. (If there is, then that is a bug in Bitcoin Core, which should be fixed)

https://developer.bitcoin.org/reference/rpc/

This example has many issues:

• It is not machine readable
• It does not indicate the version of Bitcoin Core that is documented
• It is wrong and outdated. For example https://developer.bitcoin.org/reference/rpc/getrawtransaction.html#argument-2-verbose is wrong. The type should be int, not a bool. Also it is missing verbose=3.
• It has all the problems that I mentioned in the original post.

https://wbnns.github.io/bitcoin-dev-docs/reference/rpc/index.html

This is the same as above, just rendered differently.

https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md

I am aware of the file, because I initially wrote it. It describes the concept of the interface, but it is not a specification.

https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29

This is a wiki page that lists some manually maintained software projects, so all of them have the issues I mentioned in the original post. I don't have the time right now to list all the bugs they have, but if you spend enough time you will find that they are either not a specification, or that they are unmaintained for years, or that they have implementation errors.

Edit: If there is a single software project out there, which currently correctly and fully implements the RPC API in a type-safe manner, it would be interesting to see how they approached it. But I doubt there is one?

[achow101]

Note that automatically generated RPC docs are also available for each major version at https://bitcoincore.org/en/doc/

[maflcko]

https://bitcoincore.org/en/doc/

Yes, I am aware. They are based on the help RPC reply. However, they are not machine readable, at least not in a standard format, so I don't think it helps to solve the issues highlighted in the original post.

[sipa]

I vaguely recall an issue or discussion around machine-readable API specs in the past, but I cannot find it. I believe @laanwj commented on it.

[maflcko]

I can only recall the gRPC (#16719) and CBOR (#22866) discussions, but those are serialization related, not about a standardized machine-readable interface documentation and specification.

[stickies-v]

I think this is an interesting approach to consider. I'm quite fond of OpenAPI but I'm not sure how well it would lend itself to a JSON-RPC, but perhaps it works well enough? There is a sibling project specifically focused on JSON-RPC (https://open-rpc.org/ ) but it has less tooling and support.

Would be cool if this means we could be spec-driven, build a tool that generates (server) header files from the spec so the RPC method implementations have compile time checks on parameters and responses etc.

[nflatrea]

Don't we have plethora of well documented guides and specifications on the JSON-RPC api ?

No. I'd highly doubt that there is a better source of the documentation and specification than Bitcoin Core itself. (If there is, then that is a bug in Bitcoin Core, which should be fixed)

https://developer.bitcoin.org/reference/rpc/

This example has many issues:

* It is not machine readable

* It does not indicate the version of Bitcoin Core that is documented

* It is wrong and outdated. For example https://developer.bitcoin.org/reference/rpc/getrawtransaction.html#argument-2-verbose is wrong. The type should be `int`, not a `bool`. Also it is missing verbose=3.

* It has all the problems that I mentioned in the original post.

https://wbnns.github.io/bitcoin-dev-docs/reference/rpc/index.html

This is the same as above, just rendered differently.

https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md

I am aware of the file, because I initially wrote it. It describes the concept of the interface, but it is not a specification.

https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29

This is a wiki page that lists some manually maintained software projects, so all of them have the issues I mentioned in the original post. I don't have the time right now to list all the bugs they have, but if you spend enough time you will find that they are either not a specification, or that they are unmaintained for years, or that they have implementation errors.

Edit: If there is a single software project out there, which currently correctly and fully implements the RPC API in a type-safe manner, it would be interesting to see how they approached it. But I doubt there is one?

I must have failed to understand your question, my apology 🧐
You just raised the fact that we need clear API specifications for core ?

If that so, I could be interesting to build something with OpenRPC close to OpenAPI https://open-rpc.org/

With a little bit of tweaking, we might also be able to use the Swagger UI originaly for REST API

https://petstore.swagger.io/
https://dev.to/vearutop/json-rpc-2-0-with-swagger-ui-2h3g

[laanwj]

I vaguely recall an issue or discussion around machine-readable API specs in the past, but I cannot find it. I believe @laanwj commented on it.

Much of it is already there, just not exposed. The RPCHelpMan/RPCResult structure for help handling was intended to be a start of formalizing the API, and to have some form of introspection. E.g. types are already checked against the spec while testing. It could be extended to include other data that's needed.

The same data that's used for rendering help could be returned on the RPC in a raw, machine-readable structured format. Easiest would be to define our own format, but if there is a suitable standard format that will work for JSON-RPC that's of course preferable. I don't think there was at the time I looked into it, but who knows.

[tdb3]

Concept ACK. IMHO, a formal specification (especially machine ingestible) would add value for downstream clients.

There is some great info/guidance in
https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md#versioning
https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#rpc-interface-guidelines

Maybe it's just a matter of adding content to the RPC interface guidelines section of the developer notes, but I think there is value in defining the approach that should be taken by PR authors proposing RPC changes.

For example, covering:

• Simple changes where additional fields are appended (e.g. RPC: Return permitbaremultisig and maxdatacarriersize in getmempoolinfo #29954)
• When to employ deprecatedrpc (e.g. rpc: return warnings as an array instead of just a single one #29845 (comment))
• Naming style for deprecatedrpc=<name>
• Approaches to take for significant protocol changes (e.g. Support JSON-RPC 2.0 when requested by client #27101), e.g. when to favor backward compatibility over new protocol compliance, etc.)
• Additional suggestions?