v0.35.3

[0.35.3] - 2024-04-11

Another bug-fix release that substitutes BinaryHeap for Vec in the same way that we do for BTreeMap/Set to avoid issues with the Ord constraint on the generic type (because this may be a generated type, and we don't automatically apply Ord to generated types).

v0.35.2

[0.35.2] - 2024-04-09

This is a small patch release that fixes the storage key decoding. Previously, we assumed the length of the hash of the storage prefix or entry name was 8 bytes, however it is 16.

v0.35.1

[0.35.1] - 2024-04-03

This is a small patch release that adds support for BinaryHeap in the codegen/subxt-cli
which wasn't supported by scale-typegen. This was discovered because the most recent metadata in polkadot
are now utilizing BinaryHeap.

v0.35.0

[0.35.0] - 2024-03-21

This release contains several fixes, adds no_std support to a couple of crates (subxt-signer and subxt-metadata) and introduces a few quality of life improvements, which I'll quickly cover:

Reworked light client (#1475)

This PR reworks the light client interface. The "basic" usage of connecting to a parachain now looks like this:

#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}

use subxt::lightclient::LightClient;

// Instantiate a light client with the Polkadot relay chain given its chain spec.
let (lightclient, polkadot_rpc) = LightClient::relay_chain(POLKADOT_SPEC)?;
// Connect the light client to some parachain by giving a chain spec for it.
let asset_hub_rpc = lightclient.parachain(ASSET_HUB_SPEC)?;

// Now, we can create Subxt clients from these Smoldot backed RPC clients:
let polkadot_api = OnlineClient::<PolkadotConfig>::from_rpc_client(polkadot_rpc).await?;
let asset_hub_api = OnlineClient::<PolkadotConfig>::from_rpc_client(asset_hub_rpc).await?;

This interface mirrors the requirement that we must connect to a relay chain before we can connect to a parachain. It also moves the light client specific logic into an RpcClientT implementation, rather than exposing it as a subxt::client::LightClient.

Typed Storage Keys (#1419)

This PR changes the storage interface so that, where possible, we now also decode the storage keys as well as the values when iterating over storage entries:

#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}

// Create a new API client, configured to talk to Polkadot nodes.
let api = OnlineClient::<PolkadotConfig>::new().await?;

// Build a storage query to iterate over account information.
let storage_query = polkadot::storage().system().account_iter();

// Get back an iterator of results (here, we are fetching 10 items at
// a time from the node, but we always iterate over one at a time).
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;

while let Some(Ok(kv)) = results.next().await {
    // We used to get a tuple of key bytes + value. Now we get back a
    // `kv` struct containing the bytes and value as well as the actual
    // decoded keys:
    println!("Decoded key(s): {:?}", kv.keys);
    println!("Key bytes: 0x{}", hex::encode(&kv.key_bytes));
    println!("Value: {:?}", kv.value);
}

When using the static interface, keys come back as a tuple of values corresponding to the different hashers used in constructing the key. When using a dynamic interface, keys will be encoded/decoded from the type given so long as it implements subxt::storage::StorageKey, eg Vec<scale_value::Value>.

Extrinsic Params Refinement (#1439)

Prior to this PR, one could configure extrinsic signed extensions by providing some params like so:

// Configure the transaction parameters; we give a small tip and set the
// transaction to live for 32 blocks from the `latest_block` above:
let tx_params = Params::new()
    .tip(1_000)
    .mortal(latest_block.header(), 32)
    .build();

let hash = api.tx().sign_and_submit(&tx, &from, tx_params).await?;

If you want to customize the account nonce, you'd use a different call like create_signed_with_nonce instead.

One of the downsides of the above approach is that, if you don't provide any explicit params, transactions will be immortal by default (because the signed extensions didn't have the information to do any better).

Now, with the help of a RefineParams trait, transactions will default to being mortal and living for 32 blocks unless an explicit mortality is provided as above.

One notable change is that the offline-only create_signed_with_nonce and create_partial_signed_with_nonce functions have lost the _with_nonce suffix. Since we can't discover nonce/mortality settings offline, you should now provide Params and set an explicit nonce (and mortality, if you like) when using these calls, otherwise the nonce will be set to 0 and the mortality to Immortal.

For a full list of changes, please see the following:

Added

• Reworked light client (#1475)
• no_std compatibility for subxt-signer (#1477)
• Typed Storage Keys (#1419)
• Extrinsic Params Refinement (#1439)
• Make storage_page_size for the LegacyBackend configurable (#1458)
• no_std compatibility for subxt-metadata (#1401)
• Experimental reconnecting-rpc-client (#1396)

Changed

• scale-type-resolver integration (#1460)
• subxt: Derive std::cmp traits for subxt payloads and addresses (#1429)
• CLI: Return error on wrongly specified type paths (#1397)
• rpc v2: chainhead support multiple finalized block hashes in FollowEvent::Initialized (#1476)
• rpc v2: rename transaction to transactionWatch (#1399)

Fixed

• Avoid a panic in case we try decoding naff bytes (#1444)
• Fix error mapping to wrong transaction status (#1445)
• Update DispatchError to match latest in polkadot-sdk (#1442)
• Handle errors when fetching storage keys from Unstablebackend (#1440)
• Swap type aliases around to be semantically correct (#1441)

v0.34.0

[0.34.0] - 2024-01-23

This release introduces a bunch of features that make subxt easier to use. Let's look at a few of them.

Codegen - Integrating scale-typegen and adding type aliases (#1249)

We rewrote the code generation functionality of subxt and outsourced it to the new scale-typegen crate, which serves a more general purpose.

Since a lot of types used in substrate are rich with generics, this release introduces type aliases into the generated code.
A type alias is generated for the arguments/keys or each call, storage entry, and runtime API method (#1249).

Macro - Errors for misspecified type paths (#1339)

The subxt macro provides attributes to specify custom derives, attributes, and type substitutions on a per-type basis.
Previously we did not verify that the provided type paths are part of the metadata. This is now fixed:
If you provide an invalid type path, the macro will tell you so. It also suggests similar type paths, you might have meant instead.

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale", 
    derive_for_type(path = "Junctions", derive = "Clone")
)]
pub mod polkadot {}

This gives you a compile-time error like this:

Type `Junctions` does not exist at path `Junctions`

A type with the same name is present at: 
xcm::v3::junctions::Junctions
xcm::v2::multilocation::Junctions

Macro - Recursive derives and attributes (#1379)

Previously adding derives on a type containing other types was also cumbersome, see this example:

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale",
    derive_for_type(path = "xcm::v2::multilocation::MultiLocation", derive = "Clone"),
    derive_for_type(path = "xcm::v2::multilocation::Junctions", derive = "Clone"),
    derive_for_type(path = "xcm::v2::junction::Junction", derive = "Clone"),
    derive_for_type(path = "xcm::v2::NetworkId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyPart", derive = "Clone"),
    derive_for_type(
        path = "bounded_collections::weak_bounded_vec::WeakBoundedVec",
        derive = "Clone"
    )
)]
pub mod polkadot {}

We introduced a recursive flag for custom derives and attributes that automatically inserts the specified derives on all child types:

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale",
    derive_for_type(path = "xcm::v2::multilocation::MultiLocation", derive = "Clone", recursive),
)]
pub mod polkadot {}

Subxt CLI - New features and usability improvements (#1290, #1336, and #1379)

Our CLI tool now allows you to explore runtime APIs and events (#1290). We also fully integrated with scale-typegen-description, a crate that can describe types in a friendly way and provide type examples. The output is also color-coded to be easier on the eyes. Get started with these commands:

# Show details about a runtime API call:
subxt explore --url wss://westend-rpc.polkadot.io api StakingAPI nominations_quota
# Execute a runtime API call from the CLI:
subxt explore --url wss://westend-rpc.polkadot.io api core version -e 
# Discover what events a pallet can emit:
subxt explore --url wss://westend-rpc.polkadot.io pallet Balances events

All CLI commands that take some metadata via --file or --url, can now also read the metadata directly from stdin with --file - (#1336).
This allows you to pipe in metadata from other processes like in this command chain:

parachain-node export-metadata | subxt codegen --file - | rustfmt > main.rs

Similar to the macro, the subxt codegen command can now also use recursive flags:

subxt codegen --derive-for-type xcm::v2::multilocation::MultiLocation=Clone,recursive
subxt codegen --attributes-for-type "xcm::v2::multilocation::MultiLocation=#[myerror],recursive"

Minor changes and things to be aware of

• Using insecure connections is now an explicit opt-in in many places (#1309)
• When decoding extrinsics from a block into a static type, we now return it's details (e.g. signature, signed extensions, raw bytes) alongside the statically decoded extrinsic itself (#1376)

We also made a few fixes and improvements around the unstable backend and the lightclient, preparing them for more stable usage in the future.

Added

• Errors for misspecified type paths + suggestions (#1339)
• CLI: Recursive derives and attributes (#1379)
• CLI: Explore runtime APIs and events, colorized outputs, scale-typegen integration for examples (#1290)
• Add chainflip to real world usage section of README (#1351)
• CLI: Allow using --file - to read metadata from stdin (#1336)
• Codegen: Generate type aliases for better API ergonomics (#1249)

Changed

• Return Pending rather than loop around if no new finalized hash in submit_transaction (#1378)
• Return ExtrinsicDetails alongside decoded static extrinsics (#1376)
• Improve Signed Extension and Block Decoding Examples/Book (#1357)
• Use scale-typegen as a backend for the codegen (#1260)
• Using insecure connections is now opt-in (#1309)

Fixed

• Ensure lightclient chainSpec is at least one block old (#1372)
• Typo fix in docs (#1370)
• Don't unpin blocks that may show up again (#1368)
• Runtime upgrades in unstable backend (#1348)
• Generate docs for feature gated items (#1332)
• Backend: Remove only finalized blocks from the event window (#1356)
• Runtime updates: wait until upgrade on chain (#1321)
• Cache extrinsic events (#1327)

v0.33.0

[0.33.0] - 2023-12-06

This release makes a bunch of small QoL improvements and changes. Let's look at the main ones.

Add support for configuring multiple chains (#1238)

The light client support previously provided a high level interface for connecting to single chains (ie relay chains). This PR exposes a "low level" interface which allows smoldot (the light client implementation we use) to be configured somewhat more arbitrarily, and then converted into a valid subxt OnlineClient to be used.

See this example for more on how to do this.

We'll likely refine this over time and add a slightly higher level interface to make common operations much easier to do.

Support decoding signed extensions (#1209 and #1235)

This PR makes it possible to decode the signed extensions in extrinsics. This looks something like:

let api = OnlineClient::<PolkadotConfig>::new().await?;

// Get blocks; here we just subscribe to them:
let mut blocks_sub = api.blocks().subscribe_finalized().await?;

while let Some(block) = blocks_sub.next().await {
    let block = block?;

    // Fetch the extrinsics in the block:
    let extrinsics = block.extrinsics().await?;

    // Iterate over them:
    for extrinsic in extrinsics.iter() {

        // Returns None if the extrinsic isn't signed, so no signed extensions:
        let Some(signed_exts) = extrinsic.signed_extensions() else {
            continue;
        };

        // We can ask for a couple of common values, None if not found:
        println!("Tip: {:?}", signed_exts.tip());
        println!("Nonce: {:?}", signed_exts.tip());

        // Or we can find and decode into a static signed extension type
        // (Err if we hit a decode error first, then None if it's not found):
        if let Ok(Some(era)) = signed_exts.find::<CheckMortality<PolkadotConfig>>() {
            println!("Era: {era:?}");
        }

        // Or we can iterate over the signed extensions to work with them:
        for signed_ext in signed_exts {
            println!("Signed Extension name: {}", signed_ext.name());

            // We can try to statically decode each one:
            if let Ok(Some(era)) = signed_ext.as_signed_extension::<CheckMortality<PolkadotConfig>>() {
                println!("Era: {era:?}");
            }

            // Or we can dynamically decode it into a `scale_value::Value`:
            if let Ok(value) = signed_ext.value() {
                println!("Decoded extension: {value}");
            }
        }
    }
}

See the API docs for more.

ChargeAssetTxPayment: Add support for generic AssetId

Still on the topic of signed extensions, the ChargeAssetTxPayment extension was previously not able to be used with a generic AssetId, which prohibited it from being used on the Asset Hub (which uses a MultiLocation instead). To address this, we added an AssetId type to our subxt::Config, which can now be configured.

One example of doing that can be found here.

This example uses a generated MultiLocation type to be used as the AssetId. Currently it requires a rather hideous set of manual clones like so:

#[subxt::subxt(
    runtime_metadata_path = "../artifacts/polkadot_metadata_full.scale",
    derive_for_type(path = "xcm::v2::multilocation::MultiLocation", derive = "Clone"),
    derive_for_type(path = "xcm::v2::multilocation::Junctions", derive = "Clone"),
    derive_for_type(path = "xcm::v2::junction::Junction", derive = "Clone"),
    derive_for_type(path = "xcm::v2::NetworkId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyPart", derive = "Clone"),
    derive_for_type(
        path = "bounded_collections::weak_bounded_vec::WeakBoundedVec",
        derive = "Clone"
    )
)]

This is something we plan to address in the next version of Subxt.

Change SignedExtension matching logic (#1283)

Before this release, each signed extension had a unique name (SignedExtension::NAME). We'd use this name to figure out which signed extensions to apply for a given chain inside the signed_extensions::AnyOf type.

However, we recently ran into a new signed extension in Substrate called SkipCheckIfFeeless. This extension would wrap another signed extension, but maintained its own name. It has since been "hidden" from the public Substrate interface again, but a result of encountering this is that we have generalised the way that we "match" on signed extensions, so that we can be smarter about it going forwards.

So now, for a given signed extension, we go from:

impl<T: Config> SignedExtension<T> for ChargeAssetTxPayment<T> {
    const NAME: &'static str = "ChargeAssetTxPayment";
    type Decoded = Self;
}

To:

impl<T: Config> SignedExtension<T> for ChargeAssetTxPayment<T> {
    type Decoded = Self;
    fn matches(identifier: &str, type_id: u32, types: &PortableRegistry) -> bool {
        identifier == "ChargeAssetTxPayment"
    }
}

On the whole, we continue matching by name, as in the example above, but this allows an author to inspect the type of the signed extension (and subtypes of it) too if they want the signed extension to match (and thus be used) only in certain cases.

Remove wait_for_in_block helper method (#1237)

One can no longer use tx.wait_for_in_block to wait for a transaction to enter a block. The reason for this removal is that, especially when we migrate to the new chainHead APIs, we will no longer be able to reliably obtain any details about the block that the transaction made it into.

In other words, the following sort of thing would often fail:

tx.wait_for_in_block()
  .await?
  .wait_for_success()
  .await?;

The reason for this is that the block announced in the transaction status may not have been "pinned" yet in the new APIs. In the old APIs, errors would occasionally be encountered because the block announced may have been pruned by the time we ask for details for it. Overall; having an "unreliable" higher level API felt like a potential foot gun.

That said, you can still achieve the same via the lower level APIs like so:

while let Some(status) = tx.next().await {
    match status? {
        TxStatus::InBestBlock(tx_in_block) | TxStatus::InFinalizedBlock(tx_in_block) => {
            // now, we can attempt to work with the block, eg:
            tx_in_block.wait_for_success().await?;
        },
        TxStatus::Error { message } | TxStatus::Invalid { message } | TxStatus::Dropped { message } => {
            // Handle any errors:
            println!("Error submitting tx: {message}");
        },
        // Continue otherwise:
        _ => continue,
    }
}

Subxt-codegen: Tidy crate interface (#1225)

The subxt-codegen crate has always been a bit of a mess because it wasn't really supposed to be used outside of the subxt crates, which had led to issues like #1211.

This PR tidies up the interface to that crate so that it's much easier now to programmatically generate the Subxt interface. Now, we have three properly supported ways to do this, depending on your needs:

1. Using the #[subxt] macro.
2. Using the subxt codegen CLI command.
3. Programmatically via the subxt-codegen crate.

Each method aims to expose a similar and consistent set of options.

If you were previously looking to use parts of the type generation logic to, for instance, generate runtime types but not the rest of the Subxt interface, then the https://github.com/paritytech/scale-typegen crate will aim to fill this role eventually.

That sums up the most significant changes. A summary of all of the relevant changes is as follows:

Added

• CLI: Add command to fetch chainSpec and optimize its size (#1278)
• Add legacy RPC usage example (#1279)
• impl RpcClientT for Arc<T> and Box<T> (#1277)
• RPC: Implement legacy RPC system_account_next_index (#1250)
• Lightclient: Add support for configuring multiple chains (#1238)
• Extrinsics: Allow static decoding of signed extensions (#1235)
• Extrinsics: Support decoding signed extensions (#1209)
• ChargeAssetTxPayment: Add support for generic AssetId (eg u32 or MultiLocation) (#1227)
• Add Clone + Debug on Payloads/Addresses, and compare child storage results (#1203)

Changed

• Lightclient: Update smoldot to 0.14.0 and smoldot-light to 0.12.0 (#1307)
• Cargo: Switch to workspace lints (#1299)
• Update substrate-* and signer-related dependencies (#1297)
• Change SignedExtension matching logic and remove SkipCheckIfFeeless bits (#1283)
• Update the README with the new location ...

v0.31.0

[0.31.0] - 2023-08-02

This is a small release whose primary goal is to bump the versions of scale-encode, scale-decode and scale-value being used, to benefit from recent changes in those crates.

scale-decode changes how compact values are decoded as part of #1103. A compact encoded struct should now be properly decoded into a struct of matching shape (which implements DecodeAsType). This will hopefully resolve issues around structs like Perbill. When decoding the SCALE bytes for such types into scale_value::Value, the Value will now be a composite type wrapping a value, and not just the value.

We've also figured out how to sign extrinsics using browser wallets when a Subxt app is compiled to WASM; see #1067 for more on that!

The key commits:

Added

• Add browser extension signing example (#1067)

Changed

• Bump to latest scale-encode/decode/value and fix test running (#1103)
• Set minimum supported rust-version to 1.70 (#1097)

Fixed

• Tests: support 'substrate-node' too and allow multiple binary paths (#1102)

v0.30.0

[0.30.0] - 2023-07-24

This release beings with it a number of exciting additions. Let's cover a few of the most significant ones:

Light client support (unstable)

This release adds support for light clients using Smoldot, both when compiling native binaries and when compiling to WASM to run in a browser environment. This is unstable for now while we continue testing it and work on making use of the new RPC APIs.

Here's how to use it:

use subxt::{
    client::{LightClient, LightClientBuilder},
    PolkadotConfig
};
use subxt_signer::sr25519::dev;

// Create a light client:
let api = LightClient::<PolkadotConfig>::builder()
    // You can also pass a chain spec directly using `build`, which is preferred:
    .build_from_url("ws://127.0.0.1:9944")
    .await?;

// Working with the interface is then the same as before:
let dest = dev::bob().public_key().into();
let balance_transfer_tx = polkadot::tx().balances().transfer(dest, 10_000);
let events = api
    .tx()
    .sign_and_submit_then_watch_default(&balance_transfer_tx, &dev::alice())
    .await?
    .wait_for_finalized_success()
    .await?;

At the moment you may encounter certain things that don't work; please file an issue if you do!

V15 Metadata

This release stabilizes the metadata V15 interface, which brings a few changes but primarily allows you to interact with Runtime APIs via an ergonomic Subxt interface:

// We can use the static interface to interact in a type safe way:
#[subxt::subxt(runtime_metadata_path = "path/to/metadata.scale")]
pub mod polkadot {}

let runtime_call = polkadot::apis()
    .metadata()
    .metadata_versions();

// Or we can use the dynamic interface like so:
use subxt::dynamic::Value;

let runtime_call = subxt::dynamic::runtime_api_call(
    "Metadata",
    "metadata_versions",
    Vec::<Value<()>>::new()
);

This is no longer behind a feature flag, but if the chain you're connecting to doesn't use V15 metadata yet then the above will be unavailable.

subxt-signer

The new subxt-signer crate provides the ability to sign transactions using either sr25519 or ECDSA. It's WASM compatible, and brings in fewer dependencies than using sp_core/sp_keyring does, while having an easy to use interface. Here's an example of signing a transaction using it:

use subxt::{OnlineClient, PolkadotConfig};
use subxt_signer::sr25519::dev;

let api = OnlineClient::<PolkadotConfig>::new().await?;

// Build the extrinsic; a transfer to bob:
let dest = dev::bob().public_key().into();
let balance_transfer_tx = polkadot::tx().balances().transfer(dest, 10_000);

// Sign and submit the balance transfer extrinsic from Alice:
let from = dev::alice();
let events = api
    .tx()
    .sign_and_submit_then_watch_default(&balance_transfer_tx, &from)
    .await?
    .wait_for_finalized_success()
    .await?;

Dev keys should only be used for tests since they are publicly known. Actual keys can be generated from URIs, phrases or raw entropy, and derived using soft/hard junctions:

use subxt_signer::{ SecretUri, sr25519::Keypair };
use std::str::FromStr;

// From a phrase (see `bip39` crate on generating phrases):
let phrase = bip39::Mnemonic::parse(phrase).unwrap();
let keypair = Keypair::from_phrase(&phrase, Some("Password")).unwrap();

// Or from a URI:
let uri = SecretUri::from_str("//Alice").unwrap();
let keypair = Keypair::from_uri(&uri).unwrap();

// Deriving a new key from an existing one:
let keypair = keypair.derive([
    DeriveJunction::hard("Alice"),
    DeriveJunction::soft("stash")
]);

Breaking changes

A few small breaking changes have occurred:

• There is no longer a need for an Index associated type in your Config implementations; we now work it out dynamically where needed.
• The "substrate-compat" feature flag is no longer enabled by default. subxt-signer added native signing support and can be used instead of bringing in Substrate dependencies to sign transactions now. You can still enable this feature flag as before to make use of them if needed.
  • Note: Be aware that Substrate crates haven't been published in a while and have fallen out of date, though. This will be addressed eventually, and when it is we can bring the Substrate crates back uptodate here.

For anything else that crops up, the compile errors and API docs will hopefully point you in the right direction, but please raise an issue if not.

For a full list of changes, see below:

Added

• Example: How to connect to parachain (#1043)
• ECDSA Support in signer (#1064)
• Add subxt_signer crate for native & WASM compatible signing (#1016)
• Add light client platform WASM compatible (#1026)
• light-client: Add experimental light-client support (#965)
• Add diff command to CLI tool to visualize metadata changes (#1015)
• CLI: Allow output to be written to file (#1018)

Changed

• Remove substrate-compat default feature flag (#1078)
• runtime API: Substitute UncheckedExtrinsic with custom encoding (#1076)
• Remove Index type from Config trait (#1074)
• Utilize Metadata V15 (#1041)
• chain_getBlock extrinsics encoding (#1024)
• Make tx payload details public (#1014)
• CLI tool tests (#977)
• Support NonZero numbers (#1012)
• Get account nonce via state_call (#1002)
• add #[allow(rustdoc::broken_intra_doc_links)] to subxt-codegen (#998)

Fixed

• remove parens in hex output for CLI tool (#1017)
• Prevent bugs when reusing type ids in hashing (#1075)
• Fix invalid generation of types with >1 generic parameters (#1023)
• Fix jsonrpsee web features (#1025)
• Fix codegen validation when Runtime APIs are stripped (#1000)
• Fix hyperlink (#994)
• Remove invalid redundant clone warning (#996)

v0.29.0

[0.29.0] - 2023-06-01

This is another big release for Subxt with a bunch of awesome changes. Let's talk about some of the notable ones:

A new guide

This release will come with overhauled documentation and examples which is much more comprehensive than before, and goes into much more detail on each of the main areas that Subxt can work in.

Check out the documentation for more. We'll continue to build on this with some larger examples, too, going forwards. (#968) is particularly cool as it's our first example showcasing Subxt working with Yew and WASM; it'll be extended with more documentation and things in the next release.

A more powerful CLI tool: an explore command.

The CLI tool has grown a new command, explore. Point it at a node and use explore to get information about the calls, constants and storage of a node, with a helpful interface that allows you to progressively dig into each of these areas!

Support for (unstable) V15 metadata and generating a Runtime API interface

One of the biggest changes in this version is that, given (unstable) V15 metadata, Subxt can now generate a nice interface to make working with Runtime APIs as easy as building extrinsics or storage queries. This is currently unstable until the V15 metadata format is stabilised, and so will break as we introduce more tweaks to the metadata format. We hope to stabilise V15 metadata soon; see this for more information. At this point, we'll stabilize support in Subxt.

Support for decoding extrinsics

Up until now, you were able to retrieve the bytes for extrinsics, but weren't able to use Subxt to do much with those bytes.

Now, we expose several methods to decode extrinsics that work much like decoding events:

#[subxt::subxt(runtime_metadata_path = "polkadot_metadata.scale")]
pub mod polkadot {}

// Get some block:
let block = api.blocks().at_latest().await?;

// Find and decode a specific extrinsic in the block:
let remark = block.find::<polkadot::system::calls::Remark>()?;

// Iterate the extrinsics in the block:
for ext in block.iter() {
    // Decode a specific extrinsic into the call data:
    let remark = ext.as_extrinsic::<polkadot::system::calls::Remark>()?;
    // Decode any extrinsic into an enum containing the call data:
    let extrinsic = ext.as_root_extrinsic::<polkadot::Call>()?;
}

New Metadata Type (#974)

Previously, the subxt_metadata crate was simply a collection of functions that worked directly on frame_metadata types. Then, in subxt, we had a custom metadata type which wrapped this to provide the interface needed by various Subxt internals and traits.

Now, the subxt_metadata crate exposes our own Metadata type which can be decoded from the same wire format as the frame_metadata types we used to use. This type is now used throughout Subxt, as well as in the codegen stuff, and provides a single unified interface for working with metadata that is independent of the actual underlying metadata version we're using.

This shouldn't lead to breakages in most code, but if you need to load metadata for an OfflineClient you might previously have done this:

use subxt::ext::frame_metadata::RuntimeMetadataPrefixed;
use subxt::metadata::Metadata;

let metadata = RuntimeMetadataPrefixed::decode(&mut &*bytes).unwrap();
let metadata = Metadata::try_from(metadata).unwrap();

But now you'd do this:

use subxt::metadata::Metadata;

let metadata = Metadata::decode(&mut &*bytes).unwrap();

Otherwise, if you implement traits like TxPayload directly, you'll need to tweak the implementations to use the new Metadata type, which exposes everything you used to be able to get hold of but behind a slightly different interface.

Removing as_pallet_event method (#953)

In an effort to simplify the number of ways we have to decode events, as_pallet_event was removed. You can achieve a similar thing by calling as_root_event, which will decode any event that the static interface knows about into an outer enum of pallet names to event names. if you only care about a specific event, you can match on this enum to look for events from a specific pallet.

Another reason that as_pallet_event was removed was that it could potentially decode events from the wrong pallets into what you're looking for, if the event shapes happened to line up, which was a potential foot gun.

Added as_root_error for decoding errors.

Much like we can call as_root_extrinsic or as_root_event to decode extrinsics and events into a top level enum, we've also added as_root_error to do the same for errors and help to make this interface consistent across the board.

Beyond these, there's a bunch more that's been added, fixed and changes. A full list of the notable changes in this release are as follows:

Added

• Add topics to EventDetails (#989)
• Yew Subxt WASM examples (#968)
• CLI subxt explore commands (#950)
• Retain specific runtime APIs (#961)
• Subxt Guide (#890)
• Partial fee estimates for SubmittableExtrinsic (#910)
• Add ability to opt out from default derives and attributes (#925)
• add no_default_substitutions to the macro and cli (#936)
• extrinsics: Decode extrinsics from blocks (#929)
• Metadata V15: Generate Runtime APIs (#918) and (#947)
• impl Header and Hasher for some substrate types behind the "substrate-compat" feature flag (#934)
• add as_root_error for helping to decode ModuleErrors (#930)

Changed

• Update scale-encode, scale-decode and scale-value to latest (#991)
• restrict sign_with_address_and_signature interface (#988)
• Introduce Metadata type (#974) and (#978)
• Have a pass over metadata validation (#959)
• remove as_pallet_extrinsic and as_pallet_event (#953)
• speed up ui tests. (#944)
• cli: Use WS by default instead of HTTP (#954)
• Upgrade to syn 2.0 (#875)
• Move all deps to workspace toml (#932)
• Speed up CI (#928) and (#926)
• metadata: Use v15 internally (#912)
• Factor substrate node runner into separate crate (#913)
• Remove need to import parity-scale-codec to use subxt macro (#907)

Fixed

• use blake2 for extrinsic hashing (#921)
• Ensure unique types in codegen (#967)
• use unit type in polkadot config (#943)

v0.28.0

[0.28.0] - 2022-04-11

This is a fairly significant change; what follows is a description of the main changes to be aware of:

Unify how we encode and decode static and dynamic types (#842)

Prior to this, static types generated by codegen (ie subxt macro) would implement Encode and Decode from the parity-scale-codec library. This meant that they woule be encoded-to and decoded-from based on their shape. Dynamic types (eg the subxt::dynamic::Value type) would be encoded and decoded based on the node metadata instead.

This change makes use of the new scale-encode and scale-decode crates to auto-implement EncodeAsType and DecodeAsType on all of our static types. These traits allow types to take the node metadata into account when working out how best to encode and decode into them. By using metadata, we can be much more flexible/robust about how to encode/decode various types (as an example, nested transactions will now be portable across runtimes). Additionally, we can merge our codepaths for static and dynamic encoding/decoding, since both static and dynamic types can implement these traits. Read the PR description for more info.

A notable impact of this is that any types you wish to substitute when performing codegen (via the CLI tool or #[subxt] macro) must also implement EncodeAsType and DecodeAsType too. Substrate types, for instance, generally do not. To work around this, #886 introduces a Static type and enhances the type substitution logic so that you're able to wrap any types which only implement Encode and Decode to work (note that you lose out on the improvements from EncodeAsType and DecodeAsType when you do this):

#[subxt::subxt(
    runtime_metadata_path = "/path/to/metadata.scale",
    substitute_type(
        type = "sp_runtime::multiaddress::MultiAddress<A, B>",
        with = "::subxt::utils::Static<::sp_runtime::multiaddress::MultiAddress<A, B>>"
    )
)]
pub mod node_runtime {}

So, if you want to substitute in Substrate types, wrap them in ::subxt::utils::Static in the type substitution, as above. #886 also generally improves type substitution so that you can substitute the generic params in nested types, since it's required in the above.

Several types have been renamed as a result of this unification (though they aren't commonly made explicit use of). Additionally, to obtain the bytes from a storage address, instead of doing:

let addr_bytes = storage_address.to_bytes()

You must now do:

let addr_bytes = cxt.client().storage().address_bytes(&storage_address).unwrap();

This is because the address on it's own no longer requires as much static information, and relies more heavily now on the node metadata to encode it to bytes.

Expose Signer payload (#861)

This is not a breaking change, but notable in that is adds create_partial_signed_with_nonce and create_partial_signed to the TxClient to allow you to break extrinsic creation into two steps:

1. building a payload, and then
2. when a signature is provided, getting back an extrinsic ready to be submitted.

This allows a signer payload to be obtained from Subxt, handed off to some external application, and then once a signature has been obtained, that can be passed back to Subxt to complete the creation of an extrinsic. This opens the door to using browser wallet extensions, for instance, to sign Subxt payloads.

Stripping unneeded pallets from metadata (#879)

This is not a breaking change, but adds the ability to use the Subxt CLI tool to strip out all but some named list of pallets from a metadata bundle. Aside from allowing you to store a significantly smaller metadata bundle with only the APIs you need in it, it will also lead to faster codegen, since there's much less of it to do.

Use a command like subxt metadata --pallets Balances,System to select specific pallets. You can provide an existing metadata file to take that and strip it, outputting a smaller bundle. Alternately it will grab the metadata from a local node and strip that before outputting.

Dispatch error changes (#878)

The DispatchError returned from either attempting to submit an extrinsic, or from calling .dry_run() has changed. It's now far more complete with respect to the information it returns in each case, and the interface has been tidied up. Changes include:

• For ModuleError's, instead of err.pallet and err.error, you can obtain error details using let details = err.details()? and then details.pallet() and details.error().
• DryRunResult is now a custom enum with 3 states, Success, DispatchError or TransactionValidityError. The middle of these contains much more information than previously.
• Errors in general have been marked #[non_exahustive] since they could grow and change at any time. (Owing to our use of scale-decode internally, we are not so contrained when it comes to having precise variant indexes or anything now, and can potentially deprecate rather than remove old variants as needed).
• On a lower level, the rpc.dry_run() RPC call now returns the raw dry run bytes which can then be decoded with the help of metadata into our DryRunResult.

Extrinsic submission changes (#897)

It was found by @furoxr that Substrate nodes will stop sending transaction progress events under more circumstances than we originally expected. Thus, now calls like wait_for_finalized() and wait_for_in_block() will stop waiting for events when any of the following is sent from the node:

• Usurped
• Finalized
• FinalityTimeout
• Invalid
• Dropped

Previously we'd only close the subscription and stop waiting when we saw a Finalized or FinalityTimeout event. Thanks for digging into this @furoxr!

Add at_latest() method (#900 and #904)

A small breaking change; previously we had .at(None) or .at(Some(block_hash)) methods in a few places to obtain things at either the latest block or some specific block hash.

This API has been clarified; we now have .at_latest() to obtain the thing at the latest block, or .at(block_hash) (note; no more option) to obtain the thing at some fixed block hash. In a few instances this has allowed us to ditch the async from the .at() call.

That covers the larger changes in this release. For more details, have a look at all of the notable PRs since the last release here:

Added

• added at_latest (#900 and #904)
• Metadata: Retain a subset of metadata pallets (#879)
• Expose signer payload to allow external signing (#861)
• Add ink! as a user of subxt (#837)
• codegen: Add codegen error (#841)
• codegen: allow documentation to be opted out of (#843)
• re-export sp_core and sp_runtime (#853)
• Allow generating only runtime types in subxt macro (#845)
• Add 'Static' type and improve type substitution codegen to accept it (#886)

Changed

• Improve Dispatch Errors (#878)
• Use scale-encode and scale-decode to encode and decode based on metadata (#842)
• For smoldot: support deserializing block number in header from hex or number (#863)
• Bump Substrate dependencies to latest (#905)

Fixed

• wait_for_finalized behavior if the tx dropped, usurped or invalid (#897)