INTRODUCTION

The idea comes from scannell's blog, Fuzzing for eBPF JIT bugs in the Linux kernel.

It contains three parts:

• qemu fuzzlib
• ebpf sample generator
• exception handler in the linux kernel

QEMU FUZZLIB

This module is mainly used to test the linux kernel. It uses the modified syzkaller script to generate debian buster image file and all other necessary files. The modified script adds a new normal user test without a password.

NOTE: use This create-image.sh to create the buster img: ./create-image.sh --distribution buster

This module provide an interface qemu_fuzzlib_env_setup() for the caller to initialize the fuzzing environment, the prototype of the function is:

extern struct qemu_fuzzlib_env *
qemu_fuzzlib_env_setup(char *user_name, u64 user_id, char *qemu_exec_path,
			char *bzImage_file, char *osimage_file,
			char *host_id_rsa, char *listen_ip, u32 inst_max,
			u32 idle_sec, u32 inst_memsz, u32 inst_core,
			char *env_workdir, char *guest_workdir,
			char *guest_user, char *script_file, char *c_file,
			char *sample_fname, char *fuzz_db,
			int (*db_init)(struct qemu_fuzzlib_env *),
			int (*mutate)(struct qemu_fuzzlib_env *, char *));

• user_name: the fuzzer's name, in this project, it is ebpf_fuzzer.
• user_id: default 0.
• qemu_exec_path: the binary absolute path to qemu, e.g. /usr/bin/qemu-system-x86_64.
• osimage_file: the absolute path to bzImage file.
• host_id_rsa: the id_rsa file generated by the modified script.
• listen_ip: 10.0.2.10 recommended.
• inst_max: how many qemu instances will be launched.
• idle_sec: how many seconds will wait for until an qemu instance is ready for a new sample.
• inst_memsz: the memory size for each qemu instance.
• inst_core: the core number for each qemu instance.
• env_workdir: the work directory of the fuzzing process.
• guest_workdir: the work directory of the guest, normally /tmp.
• guest_user: the user will be used to login the guest, could be test or root. We need a normal user to trigger different code paths in the kernel.
• script_file: the script file will be uploaded to the guest and be executed in the guest. Default: default_guest.sh.
• c_file: the C source file that will be uploaded to the guest and be compiled and executed in the guest to execute the sample and catch the exception of the sample process. Default: default_guest.c.
• sample_fname: the sample filename.
• fuzz_db: the fuzzing database, not used for now.
• db_init: the callback used to initialize the database.
• mutate: the callback used to generate new sample.

After the fuzzing environment is setup, the caller should call qemu_fuzzlib_env_run() to start the fuzzer.

The qemu_fuzzlib_env_run() function generates new sample and put it into an available qemu instance to execute, until no more sample is generated or no available qemu instance found after the idle_sec seconds.

EBPF SAMPLE GENERATOR

We need to focus on just one thing: the mutate() callback. This function is used to generate new sample, in this ebpf fuzzer, to generate new ebpf sample.

The scannell's blog give us a perfect guidance to generate ebpf samples. I recommend you to read the blog first.

In the current implementation, the sample's header and tail are known. We need to generate the sample body, which is filled by ebpf instructions. The instructions do several things:

• get the two bpf map pointers.
• random instructions to manipulate the INVALID_P_REG. implemented in insn_body().
• an ALU operation on CORRUPT_REG.
• read from CORRUPT_REG and write the value to STORAGE_REG.
• exit

After all instructions generated, we need to print the instructions and write them to the sample file.

insn_body()

• gen_body0(): set the SPECIAL_REG bounds.
• gen_body1(): generate bpf instructions up to max_body_insn. Six types of instructions:
  • INSN_GENERATOR_JMP: BPF_JMP.
  • INSN_GENERATOR_ALU: BPF_ALU.
  • INSN_GENERATOR_MOV: BPF_MOV.
  • INSN_GENERATOR_LD: BPF_LD_IMM64().
  • INSN_GENERATOR_NON: BPF_REG_0 = 0.
  • INSN_GENERATOR_MAX: the last insn, INVALID_P_REG = SPECIAL_REG.

EXCEPTION HANDLER IN THE LINUX KERNEL

The first time I run the fuzzer to trigger cve-2020-8835, the guest frozen: one of the kernel threads runs into an infinite loop. Check this commit: the verifier rewrote original instructions it recognized as dead code with 'goto PC-1'.

This is a good way to detect bugs in the bpf verifier.

What else?

How to run

After compiling the clib and this project, use ./ebpf_fuzzer /path/to/config 0 to startup the fuzzer.

For the bzImage file, make sure the following config options are enabled:

CONFIG_CONFIGFS_FS=y
CONFIG_SECURITYFS=y
CONFIG_E1000=y
CONFIG_BINFMT_MISC=y

When the bzImage and buster.img are ready, test the qemu first:

Launch qemu:

/usr/bin/qemu-system-x86_64 -m 2G -smp 2 -kernel /path/to/bzImage -append 'console=ttyS0 root=/dev/sda earlyprintk=serial net.ifnames=0' -drive file=/path/to/buster.img,format=raw -net user,host=10.0.2.10,hostfwd=tcp:127.0.0.1:10021-:22 -net nic,model=e1000 -enable-kvm -nographic

Communicate with the guest

ssh -q -i /path/to/buster.id_rsa -p 10021 -o 'StrictHostKeyChecking no' test@127.0.0.1 id

An example of the config file:

[
	{
		"version": "general",
		"qemu_exec_path": "/path/to/qemu-system-x86_64",
		"bzImage_path": "/path/to/bzImage",
		"osImage_path": "/path/to/buster.img",
		"rsa_path": "/path/to/buster.id_rsa",
		"idle_sec": "1800",
		"host_ip": "10.0.2.10",
		"instance_nr": "8",
		"instance_memsz": "1",
		"instance_core": "2",
		"env_workdir": "/path/to/fuzzer_workdir",
		"guest_workdir": "/tmp/",
		"guest_user": "test",
		"sample_fname": "test.c",
		"body1_len": "24",
	}
]

The body1_len is used in mutate module, it's the count of instructions to generate in gen_body1(). The larger you give, the lower valid sample rate you will get. Default value is 0x18.

FAQ

Q: When running the fuzzer, the output is 'total: 0'?
A: Try to create the buster image with ./create-image.sh --distribution buster. Check issue #1.