Integration of cargo-fuzz to improve quality assurance of Rust code in Nyx

[ThibFrgsGmz]

High level description

Nyx, being a high fidelity astrodynamics software, is built with a focus on reliability and accuracy. To further enhance the robustness of the Rust codebase, I propose the integration of fuzz testing using the cargo-fuzz package.

Fuzz testing, or fuzzing, is a quality assurance technique used to discover coding errors and security loopholes in software. By sending random, unexpected, or malformed data as inputs to the software, we can identify potential crashes, memory leaks, or other vulnerabilities that may not be caught with traditional testing methods.

Requirements

The system needs to integrate the cargo-fuzz package into the existing Rust codebase. This will involve setting up the fuzzing environment, writing fuzz targets for critical parts of the code, and running the fuzz tests.

Test plans

The fuzz tests will be run as part of the continuous integration pipeline. The success of the integration will be measured by the ability of the fuzz tests to run without causing crashes or uncovering memory leaks. Edge cases to consider include handling of extremely large inputs, malformed data, and unexpected data types.

Design

The design will involve setting up the cargo-fuzz environment, identifying parts of the code that would benefit from fuzz testing, and writing the corresponding fuzz targets. The fuzz tests will then be integrated into the continuous integration pipeline.

Algorithm demonstration

The fuzzing process does not involve a change in algorithm but rather the introduction of a new testing method. The fuzzing process will be based on the cargo-fuzz package, which has been validated and is widely used in the Rust community.

API definition

The introduction of fuzz testing does not directly affect the Nyx APIs. However, the process may indirectly lead to improvements in the APIs if the fuzz testing uncovers issues that need to be addressed.

The integration of fuzz testing can be visualized as follows:

graph LR
A[Rust Codebase] -- Fuzz Targets --> B[cargo-fuzz]

B -- Fuzz Testing --> C[Identify Crashes/Memory Leaks]
C -- Fixes --> A

[ThibFrgsGmz]

Example of GitHub workflow:

  cargo-fuzz:
    runs-on: ubuntu-latest
    name: "Fuzz Rust codebase"
    steps:
      - uses: actions/checkout@v3
      - name: "Install Rust toolchain"
        run: rustup show
      - uses: Swatinem/rust-cache@v2
      - run: cargo install cargo-fuzz
      - run: cargo fuzz build -s none

"Fuzzing" should be added to the "test" conditional compilation attributes:

#[cfg(any(test, fuzzing))]

[ThibFrgsGmz]

Here is a tiny tutorial on integrating Fuzzing into a silly Rust project with cargo-fuzz and GitHub Actions.

Step 1: Creating a Rust project

Create a new Rust project with Cargo :

cargo new calculator

Navigate to the project directory:

cd calculator

Open the src/main.rs file and replace its contents with the following code:

use calculator::calculate;

fn main() {
    let op = '+';
    let a = 5;
    let b = 3;
    match calculate(op, a, b) {
        Some(result) => println!("Result: {}", result),
        None => println!("Error: invalid operation or division by zero"),
    }
}

Create a new src/lib.rs file and add the following code:

pub fn calculate(op: char, a: i32, b: i32) -> Option<i32> {
    match op {
        '+' => Some(a + b),
        '-' => Some(a - b),
        '*' => Some(a * b),
        '/' => if b != 0 { Some(a / b) } else { None },
        _ => None,
    }
}

Open the Cargo.toml file and add the [lib] section:

[lib]
name = "calculator"
path = "src/lib.rs"

Compile and run the project to check that it works correctly:

cargo run

Step 2: Integrating cargo-fuzz

Install cargo-fuzz:

cargo install cargo-fuzz

Initialize fuzzing in your project:

cargo fuzz init

Create a new fuzzing target for the calculate function:

cargo fuzz add calculate

Open the file fuzz/fuzz_targets/calculate.rs and replace its contents with the following code:

#![no_main]
use libfuzzer_sys::fuzz_target;
use calculator::calculate;

fuzz_target!(|data: (char, i32, i32)| {
    let _ = calculate(data.0, data.1, data.2);
});

Open fuzz/Cargo.toml and add the dependency to your crate calculator:

[dependencies]
calculator = { path = "../" }

Run fuzzing:

cargo fuzz run calculate

Step 3: GitHub CI integration

Create a new .github/workflows/fuzz.yml file and add the following code:

name: Fuzzing

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  fuzzing:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Install Rust
      uses: actions-rs/toolchain@v1
      with:
        toolchain: nightly
        override: true
    - name: Install cargo-fuzz
      run: |
        cargo install cargo-fuzz
    - name: Run Fuzzing
      run: |
        cargo fuzz run calculate -- -max_total_time=180

This workflow file performs the following actions:

• It triggers the workflow on every push and pull request on the master branch.
• It defines a job named "fuzzing" which runs on a ubuntu-latest runner.
• He retrieves the source code from the repository with actions/checkout@v2.
• It installs Rust nightly with actions-rs/toolchain@v1.
• Installs cargo-fuzz.
• He runs fuzzing with a time limit of 180 seconds.