Analyze, Fix and Format features in your Rust workspace. The goal of this tool is to have this CI ready to prevent common errors with Rust features.
cargo install zepter -f --locked
zepter
- : this is the same as
run
. - run: Run a workflow from the config file. Uses
default
if none is specified. - format
- features: Format features layout and remove duplicates.
- trace: Trace dependencies paths.
- lint
- propagate-features: Check that features are passed down.
- never-enables: A feature should never enable another other.
- never-implies (
⚠️ unstable): A feature should never transitively imply another one. - only-enables (
⚠️ unstable): A features should exclusively enable another one. - why-enables (
⚠️ unstable): Find out why a specific feature is enables.
- debug: (
⚠️ unstable) just for quick debugging some stuff. - transpose (
⚠️ unstable)- dependency
- lift-to-workspace: Lifts crate dependencies to the workspace.
- dependency
To ensure that your features are in canonical formatting, just run:
zepter format features
# Or shorter:
zepter f f
The output will tell you which features are missing formatting:
Found 37 crates with unformatted features:
polkadot-cli
polkadot-runtime-common
polkadot-runtime-parachains
...
Run again with `--fix` to format them.
Re-running with --fix
/-f
:
Found 37 crates with unformatted features:
polkadot-cli
polkadot-parachain
polkadot-core-primitives
polkadot-primitives
...
Formatted 37 crates (all fixed).
Looking at the diff that this command produces; Zepter assumes a default line width of 80. For one-lined features they will just be padded with spaces:
-default = [
- "static_assertions",
-]
+default = [ "static_assertions" ]
Entries are sorted, comments are kept and indentation is one tab for your convenience 😊
- # Hi
- "xcm/std",
"xcm-builder/std",
+ # Hi
+ "xcm/std",
Let's check that the runtime-benchmarks
feature is properly passed down to all the dependencies of the frame-support
crate in the workspace of Substrate. You can use commit 395853ac15
to verify it yourself:
zepter lint propagate-feature --feature runtime-benchmarks -p frame-support --workspace
The output reveals that some dependencies expose the feature but don't get it passed down:
crate 'frame-support'
feature 'runtime-benchmarks'
must propagate to:
frame-system
sp-runtime
sp-staking
Found 3 issues and fixed 0 issues.
Without the -p
it will detect many more problems. You can verify this for the frame-support which is indeed missing the feature for sp-runtime
while sp-runtime clearly supports it 🤔.
This can be fixed by appending the --fix
flag, which results in this diff:
-runtime-benchmarks = []
+runtime-benchmarks = [
+ "frame-system/runtime-benchmarks",
+ "sp-runtime/runtime-benchmarks",
+ "sp-staking/runtime-benchmarks",
+]
The auto-fix can be configured to enable specific optional dependencies as non-optional via --feature-enables-dep="runtime-benchmarks:frame-benchmarking"
for example. In this case the frame-benchmarking
dependency would enabled as non-optional if the runtime-benchmarks
feature is enabled.
Let's say you want to ensure that specific features are never enabled by default. For this example, we will use the try-runtime
feature of Substrate. Check out branch oty-faulty-feature-demo
and try:
zepter lint never-implies --precondition default --stays-disabled try-runtime --offline --workspace
The precondition
defines the feature on the left side of the implication and stays-disabled
expressing that the precondition never enables this.
Errors correctly with:
Feature 'default' implies 'try-runtime' via path:
frame-benchmarking/default -> frame-benchmarking/std -> frame-system/std -> frame-support/wrong -> frame-support/wrong2 -> frame-support/try-runtime
Only the first path is shown in case there are multiple.
Recently there was a build error in the Substrate master CI which was caused by a downstream dependency snow
. To investigate this, it is useful to see how Substrate depends on it.
Let's find out how node-cli
depends on snow
(example on commit dd6aedee3b8d5
):
zepter trace node-cli snow
It reports that snow
is pulled in from libp2p - good to know. In this case, all paths are displayed.
node-cli -> try-runtime-cli -> substrate-rpc-client -> sc-rpc-api -> sc-chain-spec -> sc-telemetry -> libp2p -> libp2p-webrtc -> libp2p-noise -> snow
The first step is that Zepter checks that it is executed in a rust workspace. Otherwise it fails directly. Then a workflow file is located as follows:
$WORKSPACE/zepter.yaml
$WORKSPACE/.zepter.yaml
$WORKSPACE/.cargo/zepter.yaml
$WORKSPACE/.cargo/.zepter.yaml
$WORKSPACE/.config/zepter.yaml
$WORKSPACE/.config/.zepter.yaml
It uses the first file that is found and errors if none is found. Currently it not possible to overwrite the config in a sub-folder.
Note
A production example can be found in the Polkadot-SDK or in the presets
.
It is possible to aggregate the long commands into workflows instead of typing them each time. Zepter tries to locate a config file and run the default
workflow when it is bare invoked without any arguments.
Alternately, it is possible to use zepter run default
, or any other workflow name.
Config files can contain workflows like this:
workflows:
default:
- [ 'propagate-features', ... ]
- ...
It is also possible to extend previous steps:
workflows:
check:
- ...
default:
- [ $check.0, '--fix' ]
- ...
Zepter is currently being used in the Polkadot-SDK CI to spot missing features.
When these two experiments proove the usefulness and reliability of Zepter for CI application, then a more streamlined process will be introduced (possibly in the form of CI actions).
Unit tests: cargo test
UI and downstream integration tests: cargo test -- --ignored
Environment overwrites exist for the UI tests to:
OVERWRITE
: Update the UI diff locks.UI_FILTER
: Regex to selectively run UI test.KEEP_GOING
: PrintFAILED
but don't abort on the first failed UI test.
- Add feature information to the enabled deps
- Optimize
shortest_path
function - Add support for config files
- Feature sorting and deduplication
- [ ]