Loop unroll is not working as expected

[geyslan]

Description

We expect to see 600 iterations and get only 60 in find_modules_from_module_kset_list():

tracee/pkg/ebpf/c/tracee.bpf.c

Lines 799 to 800 in f61866b

	#pragma unroll
	for (int i = 0; i < MAX_NUM_MODULES; i++) {

tracee/pkg/ebpf/c/tracee.bpf.c

Line 644 in f61866b

#define MAX_NUM_MODULES 600

Steps to check it

• Get the prog ID: sudo bpftool prog list | grep lkm_seeker_kset
• Dump it: sudo bpftool prog dump xlated id ID > lkm_seeker_kset_tail_xlated
• Count a condition inside the loop: grep 'if (mod_kobj) {' lkm_seeker_kset_tail_xlated | wc -l

Output of tracee version:

v0.19.0-148-gf61866b4e

Output of uname -a:

Linux ubuntu-jammy 5.15.0-86-generic #96-Ubuntu SMP Wed Sep 20 08:23:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Linux hb 6.6.10-1-MANJARO #1 SMP PREEMPT_DYNAMIC Fri Jan  5 17:38:36 UTC 2024 x86_64 GNU/Linux

Additional details

Tested on:

• Ubuntu, clang 12.0.1-19ubuntu3
• Ubuntu and Manjaro, clang 14.0.6
• Ubuntu, clang 15.0.7
• Ubuntu, clang 16.0.6
• Ubuntu, clang 17.0.2

[OriGlassman]

You don't expect to see 600 iterations... that's the maximum amount it'll do without reporting a warning.
Generally you'd expect around the number of kernel modules loaded in your system.

[geyslan]

@OriGlassman, actually the issue is in the output bpf object not being generated with the expected unrolled iterations.

See Steps to check it in #3830 (comment).

[geyslan]

To get a full unroll, the pragma should be #pragma clang loop unroll(full). The problem with that is if we just force the unrolling a bit... clang crashes.

Try it with 61 and it works,

diff --git a/pkg/ebpf/c/tracee.bpf.c b/pkg/ebpf/c/tracee.bpf.c
index 3cec1a9ad..beef6e2ba 100644
--- a/pkg/ebpf/c/tracee.bpf.c
+++ b/pkg/ebpf/c/tracee.bpf.c
@@ -715,7 +715,7 @@ int tracepoint__sched__sched_process_fork(struct bpf_raw_tracepoint_args *ctx)
 }
 
 // number of iterations - value that the verifier was seen to cope with - the higher, the better
-#define MAX_NUM_MODULES 450
+#define MAX_NUM_MODULES 61
 
 enum
 {
@@ -871,7 +871,7 @@ statfunc int find_modules_from_module_kset_list(program_data_t *p)
     struct kobject *n = list_next_entry_ebpf(pos, entry);
     u32 flags = KSET | HIDDEN_MODULE;
 
-#pragma unroll
+#pragma clang loop unroll(full)
     for (int i = 0; i < MAX_NUM_MODULES; i++) {
         if (BPF_CORE_READ(n, name) ==
             NULL) { // Without this the list seems infinite. Also, using pos

sudo bpftool prog dump xlated id $(sudo bpftool prog list | grep lkm_seeker_kset | awk -F':' '{print $1}') | grep 'if (mod_kobj) {' | wc -l
61

... then try with 70:

fatal error: error in backend: Branch target out of insn range
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.

---

So, based on that I suspect that #pragma unroll doesn't guarantee the full expected iterations. More tests are required.