Does Falco support running via K3S on CM4? Pods error out.

[MikeCockrem]

Falco installed from Helm fails to come to the ready state on K3S running on Raspberry Pi 4 (CM4)
It seems the ebfp module can't be built, I'm hoping someone can advise me on a way forward - from the logs it seems perhaps it may not be able to find the raspbian specific kernel headers?

Following the guide here I ran:
helm upgrade falco -n falco --set tty=true falcosecurity/falco --set falcosidekick.enabled=true --set falcosidekick.config.slack.minimumpriority=notice --set falcosidekick.config.slack.webhookurl=$(base64 --decode <<< "webhook-goes-here") --set falcosidekick.config.customfields="user:redacted" --set driver.kind=ebpf

After some time this is the output:

 $ kubectl get pods -n falco
 NAME                                   READY   STATUS                  RESTARTS          AGE
 falco-falcosidekick-6894b7c676-kh8gb   1/1     Running                 0                 16h
 falco-falcosidekick-6894b7c676-2fnns   1/1     Running                 0                 16h
 falco-ppnnq                            0/2     Init:CrashLoopBackOff   200 (4m30s ago)   16h
 falco-r6nqm                            0/2     Init:CrashLoopBackOff  200 (3m47s ago)   16h
 falco-rpxlr                                0/2     Init:Error          199 (5m20s ago)   16h
 falco-z54np                            0/2     Init:Error              201 (5m18s ago)   16h_

log files are thus:

$ kubectl -n falco logs daemonsets/falco -c falco-driver-loader
Found 4 pods, using pod/falco-ppnnq
* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.36.2, driver version=6.0.1+driver, arch=aarch64, kernel release=6.1.32-v8+, kernel version=1656
* Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
* Mounting debugfs
mount: /sys/kernel/debug: permission denied.
* Filename 'falco_raspbian_6.1.32-v8+_1656.o' is composed of:
 - driver name: falco
 - target identifier: raspbian
 - kernel release: 6.1.32-v8+
 - kernel version: 1656
* Trying to download a prebuilt eBPF probe from https://download.falco.org/driver/6.0.1%2Bdriver/aarch64/falco_raspbian_6.1.32-v8%2B_1656.o
curl: (22) The requested URL returned error: 404
Unable to find a prebuilt falco eBPF probe
* Trying to compile the eBPF probe (falco_raspbian_6.1.32-v8+_1656.o)
expr: syntax error: unexpected argument '1'
make[1]: *** /lib/modules/6.1.32-v8+/build: No such file or directory.  Stop.
make: *** [Makefile:38: all] Error 2
mv: cannot stat '/usr/src/falco-6.0.1+driver/bpf/probe.o': No such file or directory
Unable to load the falco eBPF probe

Expected behaviour

Pods to come to the running state

Screenshots

Environment

• Falco version:
  0.36.2
• System info:

• Cloud provider or hardware configuration:
• OS:
  Raspbian GNU/Linux 11 (bullseye)"
• Kernel:
  Linux 6.1.32-v8+ Falco pods crashing with "free(): corrupted unsorted chunks" #1656 SMP PREEMPT Wed Jun 7 11:37:17 BST 2023 aarch64 GNU/Linux
• Installation method:
  Helm, chart version: falco-3.8.7

Additional context
I've tried adding the kernel headers and build environment.
The folder the error complains about does exist:

ls /lib/modules/6.1.32-v8+/
modules.alias      modules.builtin            modules.builtin.bin      modules.dep      modules.devname  modules.softdep  modules.symbols.bin
kernel  modules.alias.bin  modules.builtin.alias.bin  modules.builtin.modinfo  modules.dep.bin  modules.order    modules.symbols

Very cool project I'd like to start learning so I'd appreciate any pointers.
Cheers.

[Andreagit97]

ei @MikeCockrem have you tried the modern-bpf driver? It seems you have a compatible kernel driver version=6.0.1+driver

The command is the same you posted but you need to replace driver.kind=ebpf with driver.kind=modern-bpf

helm upgrade falco -n falco --set tty=true falcosecurity/falco --set falcosidekick.enabled=true --set falcosidekick.config.slack.minimumpriority=notice --set falcosidekick.config.slack.webhookurl=$(base64 --decode <<< "webhook-goes-here") --set falcosidekick.config.customfields="user:redacted" --set driver.kind=modern-bpf

[MikeCockrem]

Thanks, I've tried that out, however I get the following error:

$ kgp -n falco
NAME                                   READY   STATUS             RESTARTS        AGE
falco-falcosidekick-6894b7c676-kh8gb   1/1     Running            0               20h
falco-falcosidekick-6894b7c676-2fnns   1/1     Running            0               20h
falco-thpmh                            1/2     CrashLoopBackOff   17 (115s ago)   63m
falco-rmwbt                            0/2     CrashLoopBackOff   29 (110s ago)   63m
falco-rqtcv                            0/2     CrashLoopBackOff   29 (95s ago)    64m
falco-fp4w4                            0/2     CrashLoopBackOff   29 (90s ago)    63m

kubectl -n falco logs daemonsets/falco -c falco-driver-loader
Found 4 pods, using pod/falco-rqtcv
error: container falco-driver-loader is not valid for pod falco-rqtcv

kubectl logs -n falco falco-thpmh
Defaulted container "falco" out of: falco, falcoctl-artifact-follow, falcoctl-artifact-install (init)
Mon Jan 22 11:55:56 2024: Falco version: 0.36.2 (aarch64)
Mon Jan 22 11:55:56 2024: Falco initialized with configuration file: /etc/falco/falco.yaml
Mon Jan 22 11:55:56 2024: Loading rules from file /etc/falco/falco_rules.yaml
Mon Jan 22 11:55:56 2024: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Mon Jan 22 11:55:56 2024: Starting health webserver with threadiness 4, listening on port 8765
Mon Jan 22 11:55:56 2024: Loaded event sources: syscall
Mon Jan 22 11:55:56 2024: Enabled event sources: syscall
Mon Jan 22 11:55:56 2024: Opening 'syscall' source with modern BPF probe.
Mon Jan 22 11:55:56 2024: One ring buffer every '2' CPUs.
libpman: prog 'BPF_TRACE_RAW_TP' is not supported <-------------------------------------------<
Mon Jan 22 11:55:56 2024: An error occurred in an event source, forcing termination...
Events detected: 0
Rule counts by severity:
Triggered rules by rule name:
Error: Initialization issues during scap_init

I would guess that means support for modern-bpf is not compiled into the kernel for pi?

[Andreagit97]

uhm got it, yes the modern bpf is not supported since your kernel misses some key features needed to run it. Not sure what is causing your issue, @FedeDP any idea?

[FedeDP]

I've tried adding the kernel headers and build environment.
The folder the error complains about does exist:

Note that it complains about the build folder: /lib/modules/6.1.32-v8+/build and i don't see it being listed in your ls output.

[MikeCockrem]

I've tried adding the kernel headers and build environment.
The folder the error complains about does exist:

Note that it complains about the build folder: /lib/modules/6.1.32-v8+/build and i don't see it being listed in your ls output.

Thanks for the pointer, I messed around trying to install the kernel sources and have now come up against this error:

Trying to compile the eBPF probe (falco_raspbian_6.1.32-v8+_1656.o) 
expr: syntax error: unexpected argument '1'
make[1]: *** No targets specified and no makefile found.  Stop.
make: *** [Makefile:38: all] Error 2
mv: cannot stat '/usr/src/falco-6.0.1+driver/bpf/probe.o': No such file or directory
Unable to load the falco eBPF probe
Stream closed EOF for falco/falco-gcc4m (falco-driver-loader)

I'll try and have another look at it tomorrow.

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[poiana]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten