-
Notifications
You must be signed in to change notification settings - Fork 30
Event Schemata
Daniel Roethlisberger edited this page Jul 29, 2018
·
29 revisions
The following are all the log events currently produced by the latest
xnumon release (master branch) with at least one instance of each
object type with all possible fields set, annotated with JavaScript
syntax comments.
The structure of logged events is the same for all regular log formats
(json, json-seq, yaml, xml). Logs produced by raw mode log
destination drivers may or may not use the same schema.
Fields can be omitted, events can be suppressed, hashes can be selected, name resolution can be configured; check the default configuration for details how settings affect available fields and events.
Quick links:
- Eventcode 0: xnumon-ops
- Eventcode 1: xnumon-stats
- Eventcode 2: image-exec
- Eventcode 3: process-access
- Eventcode 4: launchd-add
{
// Version of the event schema, equivalent to LOGEVT_VERSION in logevt.h.
// Incremented with every release that modifies the schema of any event.
"version": 6,
// Timestamp of the event (not the logging) in ISO 8601 with subseconds.
"time": "2018-07-23T22:11:32.132021000Z",
// One of the eventcodes below indicating the type of event.
"eventcode": 0,
// ... fields specific to the eventcode ...
}This event is produced whenever the agent status changes; currently that's on start and stop. Automatic configuration reloading is implemented as full restart, producing a stop event with the old config and a start event with the new config. This event is meant for operational monitoring of agent status and configuration across the fleet. Except for crashed systems or crashed agents, you should see these events always in start and stop pairs bracketing the other events produced while the agent is running.
{
"version": 6,
"time": "2018-07-23T22:11:32.132021000Z",
"eventcode": 0,
// "start": Agent started.
// "stop": Agent stopped.
"op": "start",
// Build information.
"build": {
"version": "0.1.5-3-g71dfc7a",
"date": "2018-07-24",
"info": "V:GIT"
},
"config": {
// Path the config file was loaded from.
"path": "/Library/Application Support/ch.roe.xnumon/configuration.plist",
// Effective configuration after loading the config file, applying
// command line overrides and possibly downgrading settings due to
// runtime failures. See default configuration file for details.
"id": "example-0.1.5",
"launchd_mode": true, // -d command line argument
"debug": false,
"events": "0,1,2,3,4",
"stats_interval": 3600,
"kextlevel": "hash", // kextlevel will be "none" if kext failed to load
"hashes": "sha256",
"codesign": true,
"envlevel": "dyld",
"resolve_users_groups": true,
"omit_mode": false,
"omit_size": false,
"omit_mtime": false,
"omit_ctime": false,
"omit_btime": false,
"omit_sid": false,
"omit_groups": false,
"omit_apple_hashes": true,
"ancestors": "unlimited",
"logdst": "file",
"logfmt": "json",
"logoneline": true,
"logfile": "/var/log/xnumon.log",
"limit_nofile": 8192,
"suppress_image_exec_at_start": true,
"suppress_image_exec_by_ident": 3, // only counts for suppressions,
"suppress_image_exec_by_path": 1, // since they can be large lists
"suppress_image_exec_by_ancestor_ident": 0,
"suppress_image_exec_by_ancestor_path": 1,
"suppress_process_access_by_subject_ident": 4,
"suppress_process_access_by_subject_path": 0
},
// Operating system version information, equivalent to sw_vers(1).
"system": {
"name": "Mac OS X",
"version": "10.11.6",
"build": "15G22010"
}
}Logged in fixed intervals of stats_interval seconds and on agent
shutdown to capture the full run time of the agent.
The purpose of this event is to monitor various operational metrics
on agent health, audit(4) health, kext health, lost events, queue
and cache sizes, occurences and fatality of operating system bugs
in audit(4) and similar issues affecting the quality of the logs
produced. Correct interpretation of these numbers requires a good
understanding of the relevant OS facility and agent source code.
The event can also serve as an agent heartbeat.
The more fatal of the events tracked by these metrics will also
produce debug output on stderr if the debug configuration
option is enabled.
{
"version": 6,
"time": "2018-07-24T11:09:49.858115000Z",
"eventcode": 1,
// evtloop.c
// Metrics related to the main event loop.
"evtloop": {
// Number of times the global audit policy was clobbered
// by another process, most likely auditd. If this is
// greater than zero, make sure that all the required policy
// flags are set in /etc/security/audit_control, namely
// "argv" and depending on xnumon envlevel, also "arge".
"aupclobber": 1,
// Number of received audit events that had an unexpected
// event type that was not configured on the auditpipe(4).
"aueunknown": 5,
// Number of audit records for failed syscalls that were
// never actually executed and therefore ignored.
"failedsyscall": 1381,
// Number of instances of specific audit(4) bugs reported
// to Apple, and how many of those were fatal, i.e. could
// not successfully be worked around.
"radar38845422": 240,
"radar38845422_fatal": 0,
"radar38845784": 0,
"radar39267328": 18217,
"radar39267328_fatal": 0,
"radar39623812": 3066,
"radar39623812_fatal": 0,
// Number of audit events that were missing a required
// token, not including those resulting from audit(4)
// bugs already counted above. These indicate unknown
// audit(4) bugs, wrong assumptions of xnumon, or
// non-backwards-compatible audit(4) changes (in order
// or likelyhood).
"missingtoken": 0,
"oom": 0
},
// procmon.c
// Metrics related to the process monitoring core.
"procmon": {
// Number of active processes currently active.
"actprocs": 266,
// Number of executable images currently active.
// Includes ancestors.
"actexecimages": 282,
// Number of times procmon had to reconstruct a process
// image by PID from the live system instead of using
// audit(4) events. This can happen immediately after
// agent start or after a fork was missed for some other
// reason.
"liveacq": 0,
// Various different fatal event misses that could not
// be worked around. See procmon.c for details.
"miss": {
"bypid": 0,
"forksubj": 0,
"execsubj": 0,
"execinterp": 0,
"chdirsubj": 0,
"getcwd": 0
},
"oom": 0
},
// hackmon.c
// Metrics related to the interprocess hackery monitoring core.
"hackmon": {
// Number of interprocess hackery events received by hackmon.
"recvd": 18408,
// Number of interprocess hackery events actually processed
// by hackmon because they were relevant.
"procd": 3471,
"oom": 0
},
// filemon.c
// Metrics related to the file monitoring core.
"filemon": {
// Number of file events received by filemon.
"recvd": 504772,
// Number of file events actually processed by filemon
// because they were relevant.
"procd": 0,
// Number of launchd plists that could not be read because
// they already disappeared from disk again.
"lpmiss": 0,
"oom": 0
},
// kext/*.c
// Metrics related to the xnumon kernel extension.
"kext_cdevq": {
// Number of notifications waiting to be read by xnumon.
"buckets": 0,
// Number of processes currently executing or sleeping in the
// kext's KAuth handler.
"visitors": 0,
// Number of processes where xnumon failed to acknowledge the
// notification within the time limit. These processes get
// released (deferred) although xnumon may not have acquired
// all the information as configured in kextlevel.
// The kext also reports this fact in the kernel message
// buffer (dmesg).
"timeout": 0,
// Number of processes which were released (deferred) after a
// fatal error was encountered in the KAuth handler.
"error": 0,
// Number of processes which were processed (deferred) by the
// KAuth handler.
"defer": 1773,
// Number of processes which were denied execve(2) by the
// KAuth handler because their path would have exceeded the
// maximum path length.
"deny": 0
},
// procmon.c
// Metrics related to the queue for kextlevel pre-acquired images.
"prep_queue": {
// Pre-acquired images currently in the queue.
"buckets": 0,
// Number of lookups into the queue by procmon code.
"lookup": 1751,
// Number of lookups that failed to find the pre-acquired image.
// A low number of misses is normal during agent start because
// the kext and auditpipe are not in sync yet.
"miss": 2,
// Number of queue items that were dropped after expiring.
// A low number of drops is normal during agent start because
// the kext and auditpipe are not in sync yet.
"drop": 10,
// Number of queue items that were skipped during lookups.
// Happens during misses and drops, and also when events are
// received from the kext and the auditpipe in different order.
"bktskip": 164
},
// XNU bsd/security/audit/audit_pipe.c
// Metrics related to the auditpipe(4) kernel facility.
"aupi_cdevq": {
// Queue buckets currently used.
"buckets": 0,
// Maximum queue size.
"bucketmax": 1024,
// Number of audit events inserted into auditpipe(2).
"insert": 594700,
// Number of read(2) calls on the auditpipe(2) device.
// Each read(2) call returns 1 or more events.
"read": 459185,
// Number of audit events dropped in-kernel during insertion.
// Sometimes, the kernel likes to drop audit events without
// apparent reason despite the auditpipe(2) queue being empty;
// seems to happen more frequently when file related audit
// events are active. Report to Apple if you see this happen.
"drop": 63
},
// work.c
// Metrics related to the work queue.
"work_queue": {
// Number of log events waiting to be postprocessed
"buckets": 0
},
// log.c
// Metrics related to the log queue.
"log_queue": {
// Number of log events waiting to be logged
"buckets": 0,
// Logged event count by eventcode
"events": [
1,
23,
335,
0,
0
],
"errors": 0
},
// cachehash.c
// Metrics related to the cache mapping file stat to hashes.
"hash_cache": {
"buckets": 308,
"bucketmax": 12288,
"put": 308,
"get": 2037,
"hit": 1729,
"miss": 308,
"inv": 0
},
// cachecsig.c
// Metrics related to the cache mapping hashes to code signatures.
"csig_cache": {
"buckets": 296,
"bucketmax": 12288,
"put": 296,
"get": 2015,
"hit": 1719,
"miss": 296,
"inv": 0
},
// cacheldpl.c
// Metrics related to the cache tracking known launchd plists.
"ldpl_cache": {
"buckets": 565,
"bucketmax": 1536,
"put": 565,
"get": 0,
"hit": 0,
"miss": 0,
"inv": 0
}
}Logged when a process successfully replaces its executable image by
calling one of the execve(2) family or posix_spawn(2) syscalls.
{
"version": 6,
"time": "2018-07-24T12:35:48.582000000Z",
"eventcode": 2,
// Arguments as seen by execve(2) or posix_spawn(2).
// argv[0] can be absolute or relative.
"argv": [
"./chkcs.signed",
"-v",
"/bin/sh"
],
// Environment variables as seen by execve(2) or posix_spawn(2).
// Presence depends on envlevel. If no variables match `DYLD_*`
// in envlevel=dyld, then "env" is absent. Also, if the global
// audit policy flag "arge" gets clobbered by an incorrectly
// configured auditd, the "env" field will be absent until xnumon
// has detected and fixed the clobbered audit policy.
"env": [
"DYLD_LIBRARY_PATH=/foo/bar",
"DYLD_INSERT_LIBRARIES=/foo/bar/baz.dylib"
],
// Current working directory at the time of exec.
"cwd": "/Users/jdoe/xnumon",
// New image that is being executed by the subject.
// For shebang script execution, this is the interpreter executable.
"image": {
// Absolute path to the image file on disk.
// This is the only field guaranteed to be present.
"path": "/Users/jdoe/xnumon/chkcs.signed",
// File mode including type bits, in octal.
// Absent if file stat could not be acquired.
"mode": "0100755",
// File owner uid and resolved user name.
// Except for uid -1, the uid is always unsigned.
// Absent if file stat could not be acquired.
"uid": 501,
"uname": "jdoe",
// File group gid and resolved group name.
// Except for gid -1, the gid is always unsigned.
// Absent if file stat could not be acquired.
"gid": 20,
"gname": "staff",
// File size.
// Absent if file stat could not be acquired.
"size": 140128,
// File modification time, change time and birth time.
// Absent if file stat could not be acquired.
"mtime": "2018-07-24T12:28:07.000000000Z",
"ctime": "2018-07-24T12:28:07.000000000Z",
"btime": "2018-07-24T12:28:04.000000000Z",
// File hashes.
// Absent if hashes could not be acquired.
"md5": "59151c65075f96f9519fcadd35502593",
"sha1": "f54dfa8dad7cdf2de655babf8d5ede4e30ea62e2",
"sha256": "301da5e482f005f9726fc35e073b9154c2773031f3264f055c4b16d42d0db131",
// Code signature status, one of:
// "unsigned": Executable does not carry a signature.
// "good": Signature is good and anchored at a trusted root.
// "untrusted": Signature is present but not anchored at a trusted root.
// "bad": Signature failed to validate.
// "error": Error during signature validation.
// Absent if code signature could not be acquired.
"signature": "good",
// Code signature origin, present only if signature is "good", one of:
// "system": Apple System, part of the OS or otherwise Apple proper.
// "appstore": Mac App Store, third party code distributed by Apple.
// "devid": Developer ID, third party code distributed directly.
// "generic": Apple Generic, signed by Apple-issued certificate but
// neither Apple proper, Mac App Store nor Developer ID.
// You should not see this unless Apple changes their PKI
// architecture.
// "trusted": Trusted CA, third party code using non-Developer-ID
// code signing certificate issued by a CA trusted by the
// system's trust database for code signing.
// Field is absent if signature != good or absent.
"origin": "devid",
// CDHash that was signed.
// Field is absent if no CDHash was found.
"cdhash": "6c812f10263c71141583e581e1f9105a9b474e0f",
// Ident string embedded in the signature.
// Field is absent if no ident string was found.
"ident": "ch.roe.chkcs",
// Team ID embedded in the signature.
// Field is absent if no team ID was found.
"teamid": "C9BFEG985N",
// Certificate Common Name that produced the signature.
// Field is only present when relevant (devid, trusted, generic).
"certcn": "Developer ID Application: Daniel Roethlisberger (C9BFEG985N)"
},
// For script execution, this is the script file containing the shebang.
// Has the same fields as image above, except no code signature.
// Only present for shebang-based script execution.
"script": {
"path": "/path/to/script.sh",
"mode": "0100755",
"uid": 501,
"uname": "jdoe",
"gid": 20,
"gname": "staff",
"size": 31337,
"mtime": "2018-07-24T12:28:07.000000000Z",
"ctime": "2018-07-24T12:28:07.000000000Z",
"btime": "2018-07-24T12:28:04.000000000Z",
"md5": "d41d8cd98f00b204e9800998ecf8427e",
"sha1": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
},
// Subject process that is executing a new executable image.
"subject": {
// Present if this process was reconstructed by querying the kernel by PID
// instead of tracking fork and exec. The field is absent if false.
"reconstructed": true,
// Process ID.
// Note that these are not unique per system and can repeat after a while.
"pid": 87266,
// Audit user ID and resolved user name as reported by audit(4).
"auid": 501,
"auname": "jdoe",
// Effective user ID and resolved effective user name.
"euid": 501,
"euname": "jdoe",
// Effective group ID and resolved effective group name.
"egid": 20,
"egname": "staff",
// Real user ID and resolved real user name.
"ruid": 501,
"runame": "jdoe",
// Real group ID and resolved real group name.
"rgid": 20,
"rgname": "staff",
// Audit session ID reported by audit(4).
"sid": 100007,
// TTY device the process is associated with.
// Absent if none or /dev/null.
"dev": "/dev/console",
// Source IPv4 or IPv6 address the TTY is connected to.
// Absent if none.
"addr": "192.0.2.23",
// Timestamp when the process was created by its parent.
// Absent if unknown.
"fork_time": "2018-07-24T12:35:48.549000000Z",
// Executable image active in the process at the time it called exec.
// In other words, the direct ancestor image of the image being execed.
"image": {
// Timestamp the current executable image was activated.
// Absent if the last exec was not actually seen, such as when the
// process was reconstructed by PID from the live system.
"exec_time": "2018-07-24T09:39:36.791000000Z",
// The pid at the time of executing this image.
"exec_pid": 84105,
// The following are the reduced set of image fields, see the
// above description of the image object for documentation.
"path": "/bin/zsh",
"md5": "52d52aa8579eaef58addc0757f9c97d9",
"sha1": "7e2e42198ad292d0a0c5cd765eb17f108c3c49b2",
"sha256": "72508fe3cef985ca91b766727c4d9a6374f216a8b69a13ce41677f2e0400eacd",
// Ident and teamid only present here if the signature was good.
"ident": "com.apple.zsh",
"teamid": "59GAB85EFG",
// Script object shown for completeness, only present for script execs.
"script": {
"path": "/path/to/script.sh",
"md5": "d41d8cd98f00b204e9800998ecf8427e",
"sha1": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
},
// All the previous image executions that led to the current process image,
// tracked across fork and exec, all the way back to launchd (pid 1).
// The full ancestry is therefore: image -> subject.image -> subject.ancestors.
// For processes that were already running when xnumon launched, this is
// based on the ppid.
// Each has the same fields as subject.image, see the documentation above.
"ancestors": [
{
"exec_time": "2018-07-24T09:39:36.758000000Z",
"exec_pid": 84105,
"path": "/bin/bash",
"ident": "com.apple.bash",
// Reduced script objects are also shown for ancestors.
"script": {...}
},
{
"exec_time": "2018-07-24T09:39:36.637000000Z",
"exec_pid": 84104,
"path": "/usr/bin/login",
"ident": "com.apple.login"
},
{
// This process was already running when xnumon started, so no exec_time
// and potentially missing execs from this point on.
"exec_pid": 76175,
"path": "/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal",
"ident": "com.apple.Terminal"
},
{
"exec_pid": 1,
"path": "/sbin/launchd",
"ident": "com.apple.xpc.launchd"
}
]
}
}Logged when a process successfully accesses another process by
using the task_for_pid Mach API or the ptrace(2) syscall,
regardless of whether the access results in code injection or
other manipulation of the executable state, or not.
{
"version": 6,
"time": "2018-07-21T12:55:33.482000000Z",
"eventcode": 3,
// Method of interprocess access: "task_for_pid" or "ptrace"
"method": "task_for_pid",
// The process that was accessed by the subject process.
// Same fields as eventcode 2 subject, see above.
"object": {
"pid": 13074,
"auid": 501,
"auname": "jdoe",
"euid": 501,
"euname": "jdoe",
"egid": 20,
"egname": "staff",
"ruid": 501,
"runame": "jdoe",
"rgid": 20,
"rgname": "staff",
"sid": 100007,
"fork_time": "2018-07-21T08:55:17.925000000Z",
"image": {
"exec_time": "2018-07-21T08:55:17.925000000Z",
"exec_pid": 13074,
"path": "/Applications/Firefox.app/Contents/MacOS/plugin-container.app/Contents/MacOS/plugin-container",
"sha256": "6cf2390401f8c4008c9bf8b56f46fc07aa16eb5266c0599a98158656308473e0",
"ident": "org.mozilla.plugincontainer",
"teamid": "43AQ936H96"
},
"ancestors": [
{
"exec_time": "2018-07-21T07:41:33.661000000Z",
"exec_pid": 12270,
"path": "/Applications/Firefox.app/Contents/MacOS/firefox",
"sha256": "a02d4910bfac91381b6106d7295de73e7b7708c7e00d8de38ec2eee818eab175",
"ident": "org.mozilla.firefox",
"teamid": "43AQ936H96"
},
{
"exec_time": "2018-07-21T07:41:33.658000000Z",
"exec_pid": 12270,
"path": "/usr/libexec/xpcproxy",
"ident": "com.apple.xpc.proxy"
},
{
"exec_pid": 1,
"path": "/sbin/launchd",
"ident": "com.apple.xpc.launchd"
}
]
},
// The process who has accessed the object process.
// Same fields as eventcode 2 subject, see above.
"subject": {
"pid": 25339,
"auid": -1,
"euid": 0,
"euname": "root",
"egid": 0,
"egname": "wheel",
"ruid": 0,
"runame": "root",
"rgid": 0,
"rgname": "wheel",
"sid": 100000,
"dev": "/dev/console",
"fork_time": "2018-07-21T12:55:33.480000000Z",
"image": {
"exec_time": "2018-07-21T12:55:33.480000000Z",
"exec_pid": 25339,
"path": "/bin/launchctl",
"ident": "com.apple.xpc.launchctl"
},
"ancestors": [
{
"exec_pid": 75,
"path": "/Library/Little Snitch/Little Snitch Daemon.bundle/Contents/MacOS/Little Snitch Daemon",
"sha256": "e85eff9e8890f9a2b550afb0d1e6c40933f10e4b5ae62fcc9c2430212308856a",
"ident": "at.obdev.LittleSnitchDaemon",
"teamid": "MLZF7K7B5R"
},
{
"exec_pid": 1,
"path": "/sbin/launchd",
"ident": "com.apple.xpc.launchd"
}
]
}
}Logged when a process adds or modifies a Launch Agent or Launch Daemon plist file in the global or per-user launchd configuration directories.
{
"version": 6,
"time": "2018-07-23T14:46:24.324000000Z",
"eventcode": 4,
// Information on the launchd plist file that was written.
"plist": {
// Absolute path to the plist file.
"path": "/Library/LaunchDaemons/com.citrix.ctxusbd.plist"
},
// Information on the program configured in the plist file.
"program": {
// Path to the program that will be launched by launchd.
// Always present and always an absolute path, regardless of
// whether the plist has a Program key or not.
"path": "/Library/Application Support/Citrix Receiver/ctxusbd",
// The contents of the ProgramArguments array in the plist,
// which is the argv that launchd will use to launch it.
// Absent if ProgramArguments is not used in the plist.
"argv": [
"ctxusbd"
]
},
// Process that wrote the new or modified plist file.
// Same as eventcode 2 subject, see above.
"subject": {
"pid": 64469,
"auid": -1,
"euid": 0,
"euname": "root",
"egid": 0,
"egname": "wheel",
"ruid": 0,
"runame": "root",
"rgid": 0,
"rgname": "wheel",
"sid": 100000,
"dev": "/dev/console",
"fork_time": "2018-07-23T14:46:24.311000000Z",
"image": {
"exec_time": "2018-07-23T14:46:24.311000000Z",
"exec_pid": 64469,
"path": "/System/Library/PrivateFrameworks/PackageKit.framework/Versions/A/Resources/shove",
"ident": "com.apple.shove"
},
"ancestors": [
{
"exec_pid": 291,
"path": "/System/Library/PrivateFrameworks/PackageKit.framework/Versions/A/Resources/installd",
"ident": "com.apple.installd"
},
{
"exec_pid": 1,
"path": "/sbin/launchd",
"ident": "com.apple.xpc.launchd"
}
]
}
}