fd.name some times contain bogus <UNKNOWN> part in "Write below root" event #3120
Comments
|
Thanks @petterreinholdtsen: Just to clarify
is truly sometimes, not all the times? Would you mind observing again with the following additional output fields? In addition, is it some new issues or have you observed this before in Falco? Plus could you check if you have a decent amount of dropped |
|
[Melissa Kilby]
is truly sometimes, not all the times?
Yes. Here is an example with // instead of <UNKNOWN>:
12:26:02.996402180: Error File below / or /root opened for writing
(file=//cf_lock.lmdb.lock evt_type=openat user=root user_uid=0
user_loginuid=-1 process=cf-agent proc_exepath=/usr/bin/cf-agent
parent=cf-execd command=cf-agent terminal=0 exe_flags=O_RDWR
container_id=host container_name=host)
This is on the same machine.
Would you mind observing again with the following additional output
fields? `%fd.nameraw %fs.path.name %fs.path.nameraw %proc.cwd`
(https://falco.org/docs/reference/rules/supported-fields/) -> are all
of these fields missing?
I will see what I can do. I do not have direct access to any of the
problematic machines, only the logs from them, but will try to figure
out a way to test a modified setup one one of the machines with this
behavior.
In addition, is it some new issues or have you observed this before in
Falco? Plus could you check if you have a decent amount of dropped
`open*` syscalls leveraging
https://falco.org/docs/metrics/falco-metrics/, see also
https://falco.org/docs/troubleshooting/dropping/.
In total for the last event bulk I see 18 "Falco internal: syscall event
drop" events across 5 machines and 8000 events. All of them happened on
one of the machines with the <UNKNOWN> path, but the <UNKNOWN> path show
up on more than one machine.
…--
Happy hacking
Petter Reinholdtsen
|
|
@petterreinholdtsen would it be possible to instead use the new https://falco.org/docs/metrics/falco-metrics/ framework as it supports monotonic counters? That way we know more accurately how many dropped events there are since the beginning of launching Falco and more. Thanks! Looking forward to hearing back from you in regards to the other fields |
|
[Melissa Kilby]
@petterreinholdtsen would it be possible to instead use the new
https://falco.org/docs/metrics/falco-metrics/ framework as it supports
monotonic counters?
I am reluctant to add this to all our hosts, as we try to reduce the
noise and number of irrelevant events in the log, and the metrics seem
add regular quite a lot of "non-events' to the log.
Looking forward to hearing back from you in regards to the other
fields `%fd.nameraw %fs.path.name %fs.path.nameraw %proc.cwd`.
This change rolled out this weekend, and here are a few example log
events, one with <UNKNOWN> and one with a more path like file value.
Error File below / or /root opened for writing
(file=<UNKNOWN>cf_lock.lmdb.lock nameraw=cf_lock.lmdb.lock
pathname=/etc/sssd/cf_lock.lmdb.lock pathnameraw=cf_lock.lmdb.lock
proccwd=/etc/sssd/ evt_type=openat user=root user_uid=0
user_loginuid=-1 process=cf-agent proc_exepath=/usr/bin/cf-agent
parent=cf-execd command=cf-agent terminal=0 exe_flags=O_RDWR
container_id=host container_name=host)
Error File below / or /root opened for writing
(file=//cf_lock.lmdb.lock nameraw=cf_lock.lmdb.lock
pathname=/var/cfengine/state/cf_lock.lmdb.lock pathnameraw=cf
_lock.lmdb.lock proccwd=/var/cfengine/state/ evt_type=openat user=root
user_uid=0 user_loginuid=-1 pr ocess=cf-agent
proc_exepath=/usr/bin/cf-agent parent=cf-execd command=cf-agent
terminal=0 exe_flags=O _RDWR container_id=host container_name=host)
…--
Happy hacking
Petter Reinholdtsen
|
|
[Petter Reinholdtsen]
This change rolled out this weekend, and here are a few example log
events, one with <UNKNOWN> and one with a more path like file value.
Perhaps the relevant difference is the file permission of the cwd
directory?
drwx------. 1 root root 36 jan. 12 01:00 /etc/sssd/
drwxr-xr-x. 1 root root 318 jan. 18 12:37 /var/cfengine/
Even with the latter, '//' seem like the wrong path.
…--
Happy hacking
Petter Reinholdtsen
|
|
Re https://falco.org/docs/metrics/falco-metrics/ you can adjust the timer aka how often you want to emit a log. Just FYI.
Thank you for deploying the additional output fields 🙏 ! @LucaGuerra and @FedeDP given that @petterreinholdtsen this needs a patch in Falco's libs, ETA unknown and we should consider releasing a patch release. We will keep you updated. @falcosecurity/falco-maintainers @leogr |
|
Given your examples above #3120 (comment) effectively the derivation of CC @FedeDP who shared a similar assessment. In summary, I am not sure if there is something to fix easily. At the same time if @petterreinholdtsen did you observe a regression between release or is this just a new observation? Furthermore, could you check for example what the true location of Code in question:
More references: |
Some times the events from sandbox rule "Write below root" contain a incomplete path, making it hard to know the location of the file in question. This is an example message:
"14:27:16.402853438: Error File below / or /root opened for writing (file=<UNKNOWN>cf_lock.lmdb.lock evt_type=openat user=root user_uid=0 user_loginuid=-1 process=cf-agent proc_exepath=/usr/bin/cf-agent parent=cf-execd command=cf-agent terminal=0 exe_flags=O_RDWR container_id=host container_name=host)",
The process triggering the event is cfengine with its lock file, and in other similar events the fd.name value is "//cf_lock.lmdb.lock", so I suspect the real value of the <UNKNOWN> part should be "/".
This is running on a RHEL 9.3 machine using the "modern" ebpf probe, with a setup where all rule sets from https://github.com/falcosecurity/rules/ are enabled. The machine uses cfengine to keep the setup correct and up to date.
I expected fd.name to contain a full path to the opened file, and suspect there is a bug somewhere causing the <UNKNOWN> part.
The text was updated successfully, but these errors were encountered: