Releases: osquery/osquery
5.4.0
5.4.0
Representing commits from 15 contributors! Thank you all.
New Features
- We're extending macOS Endpoint Security to include File Integrity monitoring. Check out the new
es_process_file_events
table. (#7579) - Add Docker build scripts and configuration (#7619)
Deprecation Notices
Table Changes
- New Table:
es_process_file_events
for macOS Endpoint Security based FIM (#7579) - New Table:
password_policy
table for macOS (#7594) - New Table:
windows_update_history
(#7407) - Add
memory_available
to linuxmemory_info
table (#7669) - Port the
cpu_info
table to linux (#7499) - Remove the
lldp_neighbors
table (#7664) - Update
deb_packages
table to not sisplay arch info in the package name (#7638) - Update
hardware_model
in thesystem_info
table on Apple M1 machines to report correctly (#7662) - Update
shared_resources
table to add type names, fix type/maximum_allowed handling (#7645)
Under the Hood improvements
- Expand env vars before trying to enumerate crashes in
windows_crashes
table (#7391) - Implement a split and trim function using std::string_view (#7636)
- Improve scheduled query denylisting and scheduler shutdown (#7492)
- Prevent CLI_FLAGs to be set via config (#7561)
- Remove unnecessary string copy (#7625)
Bug Fixes
- Add linwin to list of supported PLATFORM_DIRS (#7646)
- Fix AWS certificate verification failing on all services (#7652)
- Fix MBCS support on Windows (#7593)
- Fix
local_timezone
column in thetime
table on Windows (#7656) - Fix
system_info
table to support unicode on Windows (#7626) - Fix multiple Yara leaks (#7615)
- Fix std::bad_alloc on pci_devices on Apple Silicon macs (#7648)
- Fix tables spec files to specify
linux
and notposix
(#7644) - Fix thrift server shutting down when dropping privileges (#7639)
Documentation
- CHANGELOG 5.3.0 (#7575)
- Exclude
spec/example.table
when generating documentation (#7647) - Fix a UUID typo in the
disk_encryption
table (#7608) - Fix spelling of the word "owned" (#7630)
- Fix typo in FIM docs for Windows (#7676)
- Update the "new release" issue template (#7607)
- clarify browser_plugins table is referencing basically unsupported CNPAPI tech (#7651)
Build
- Add an option to build with the leak sanitizer (#7609)
- Fix check for PIE support (#7234)
- Fix SchedulerTests.test_scheduler_drift_accumulation flakyness (#7613)
- Improve config parsing and osqueryfuzz-config performance (#7635)
- Initialize users and groups services on all tests that need them (#7620)
- ci: Update osquery-packaging commit to the latest one (#7667)
- cmake: Add an option to enable or disable using ccache (#7671)
- libs: Update OpenSSL to version 1.1.1o (#7629)
- libs: Update OpenSSL to version 1.1.1q (#7674)
- libs: Update libarchive to version 3.6.1 (#7654)
- libs: Update sqlite to version 3.38.5 (#7628)
5.3.0
5.3.0
osquery 5.3.0 brings several table improvements and bugfixes.
Worth mentioning also the deprecation of the smart_drive_info
table
and the new warning added when incorrectly configuring a CLI only flag
via the config file. In the next release CLI only flags will not be
configurable through the config file or refresh anymore.
This release represents commits from 15 contributors! Thank you all.
Deprecation Notices
- Deprecate unmaintainable legacy table,
smart_drive_info
#7464
New Features
- Add the option
tls_disable_status_log
to prevent status logs from being sent via TLS #7550 - Add SQLite function
in_cidr_block
to check if IPv4/v6 addresses are within the supplied CIDR block #7563
Table Changes
- Add the
admindir
column to thedeb_packages
table to parse package databases on different paths #7549 - Implement and fix
wifi_networks
on macOS Big Sur and newer #7503 - Add windows/darwin support to
npm_packages
#7536 - Move
apt_sources
andyum_sources
tables to linux only #7537 - Add homebrew paths to the
python_packages
table #7535 - Mark
wall_time
column inosquery_schedule
as hidden #7501 - Add new metrics and improve description of existing ones in
osquery_schedule
#7438 - Add the
mirrorlist
column in the tableyum_sources
#7479 - Implement
output_size
forosquery_schedule
#7436 deb_packages
table: Use additional instead of index for theadmindir
column #7573certificates
table: Add Linux support #7570- Add
translated
column toprocesses
table to indicate whether the process is running under Apple Rosetta #7507 - Add the "internet password" type to the macOS
keychain_items
table #7576 - Add
original filename
column tofile
table on Windows #7156
Bug Fixes
- Fix watchdog not killing unhealthy worker/extension fast enough #7474
- Fix the
test_http_server.py
--persist
option #7497 - Update
profile.py --leaks
for python3 #7534 - Fixes osquery tls connections to aws kinesis when tls_server_certs is set #7450
- Fix parsing issue when a backslash as the last character on sudoers file line #7440
- Change the JSON of the results coming from an event scheduled query to an array #7434
- Fix globToRegex truncating UTF16 characters #7430
- Prevent hanging when the WMI server does not respond #7429
- Fix
python_packages
table so that it lists python packages from any user Python installations #7414 - Set string size limit on thrift protocol factory to prevent a crash #7484
- Fix driver image path in
drivers
table #7444 - Do not remove nonblocking flag when reading "special" files, to prevent hangs #7530
- Fix crash due to interaction between distributed and config plugin #7504
- bpf: Disable the BPF publisher in case of error #7500
- Warn about setting CLI_FLAGs in the config #7583
- Explicitly set context for the tables reading utmpx databases #7578
- bpf: Improve socket event handling #7446
- certificates: Refactor the OpenSSL utilities #7581
- Fix shared_resources accessing uninitialized variables #7600
Under the Hood improvements
- Implement a performant cache for users and groups on Windows #7516
- Replace WmiRequest constructor with static factory method to improve error handling and prevent crashes #7489
- Remove redundant string conversion #7603
Build
- Fix DebPackages.test_sanity test when the
size
column is empty #7569 - libs: Update libdpkg from version v1.19.0.5 to v1.21.7 #7549
- CI: Restore some release checks #7558
- Prevent ebpfpub linking against the system zlib #7557
- Fix mdfind.test_sanity flaky behavior #7533
- Enable fuzzing and Asan on Windows, enable Asan on macOS #7470
- Update cppcheck to version 2.6.3 and skip analysis for third party code #7455
- Change
cpu_info
test to expect at least one socket, not just one #7490 - Fix third party libraries flags leaking to osquery targets #7480
- Add third party libraries target #7467
- Do not run clang-tidy on third party libraries #7432
- CI: Create github workflow target to gate mergeability #7427
- Fix some warnings about unrecognized special characters in the Windows event log test #7478
- Change where the macOS Info.plist is generated #7566
- Add OSQUERY_ENABLE_THREAD_SANITIZER to optionally enable TSan #6997
- Add an option to specify a path to the openssl archive #7559
- packs: Update reverse shell query pack to check for a valid remote_port #7567
- Remove the test_daemon_sighup test #7584
Documentation
5.2.3
5.2.3
Osquery 5.2.3 is a security update that focuses on updating some third-party libraries
which contained CVEs that could affect osquery.
Additionally some other third-party libraries and tables have been dropped,
since they were not maintained or considered safe anymore.
Deprecation Notices
- Remove the
shortcut_files
table #7545 - Remove the ssdeep library and remove its support in the
hash
table #7520 - Remove the libelfin library and elf parsing tables #7510
Hardening
5.2.2
Osquery 5.2.2 brings native Apple Silicon (M1) support to the macOS platform. It also represents a comprehensive review and update of our third-party dependencies. To support this work, the developer docs have been updated, as have several parts of the build system
This release represents commits from 24 contributors! Thank you all.
New Features
- Apple Silicon support (#7330)
Deprecation Notices
- The
cpuid
table is x86 only. See #7462 - The
smart_drive_info
table has been deprecated, and is not included in the m1 builds. See #7464 - The
lldp_neighbors
table has been deprecated, and is not included in the m1 builds. See #7463
Table Changes
- Update
time
table to always reflect UTC values (#7276, #7460, #7437) - Hide the deprecated
antispyware
column inwindows_security_center
(#7411) - Add
windows_firewall_rules
table for windows (#7403)
Bug Fixes
- Update the ATC table
path
column check to be case insensitive (#7442) - Fix a crash introduced by 5.2.0 when Yara uses its own strutils functions (#7439)
- Fix
user_time
andsystem_time
unit in processes table on M1 (#7473)
Documentation
Build
- Update sqlite to version 3.37.0 (#7426)
- Fix linking of thirdparty_sleuthkit (#7425)
- Fix how we disable tables in the fuzzer init method (#7419)
- Prevent running discovery queries when fuzzing (#7418)
- Add BOOST_USE_ASAN define when enabling Asan (#7469)
- Removing unnecessary macOS version check (#7451)
- Fix submodule cache for macOS CI runner (#7456)
- Add osquery version to macOS app bundle Info.plist (#7452)
- libs: Update OpenSSL to verion 1.1.1l (#7330)
- libs: Update augeas to version 1.12.0 (#7330)
- libs: Update aws-sdk to version 1.9.116 (#7330)
- libs: Update boost to version 1.77 (#7330)
- libs: Update gflags to 2.2.2 (#7330)
- libs: Update glog to version 0.5.0 (#7330)
- libs: Update googletest to version 1.11.0 (#7330)
- libs: Update libarchive to version 3.5.2 (#7330)
- libs: Update libcap to version 1.2.59 (#7330)
- libs: Update libmagic to version 5.40 (#7330)
- libs: Update librdkafka to version 1.8.0 (#7330)
- libs: Update libxml2 to version 2.9.12 (#7330)
- libs: Update linenoise-ng to the latest commit (#7330)
- libs: Update lzma to version 5.2.5 (#7330)
- libs: Update rocksdb to version 6.22.1 (#7330)
- libs: Update sleuthkit to version 4.11.0 (#7330)
- libs: Update ssdeep-cpp to the latest commit (d8705da) (#7330)
- libs: Update thrift to version 0.15.0 (#7330)
- libs: Update yara to version 4.1.3 (#7330)
- libs: Update zstd to version 1.4.0 (#7330)
5.2.1
5.2.0
5.1.0
Representing commits from 20 contributors! Thank you all.
Note: The linux .tar.gz includes debugging symbols. This may be larger than you expect
New Features
- Allow custom cpu limit duration for the watchdog (#7348)
- Support custom endpoints for AWS Kinesis and Firehose. (#7317)
Table Changes
- Add
docker_container_envs
table for access to docker container environment (#7313) curl
table now returns peer certificates even if the TLS handshake does not complete (#7349)
Under the Hood improvements
- Allow tests and SDK to reset dispatcher state (#7372)
- Avoid string copies when looping through cron search dirs (#7331)
- Respect
read_max
flag when hashing using ssdeep (#7367)
Bug Fixes
- Detect when an extension has not started correctly on Windows (#7355)
- Fix crash #7353 when osquery captures kill syscall when not subscribed to them (#7354)
- Fix crash in AuditdNetlinkReader::configureAuditService when audit_add_rule_data returns an error (#7337)
- Fix crash when
windows_security_products
errors out (#7401) - Fix for #7394 where cleanup of some event tables never occures (#7395)
- Improve BPF publisher reliability (#7302)
- Lower log level of "executing distributed query" (#7386)
- Reduce excessive log messages from
authorized_keys
table implementation (#7318)
Documentation
- Add 5.0.1 CHANGELOG (#7284)
- Fix typo in Everything in SQL docs (#7338)
- Fix typo in SQL docs (#7376)
- Update GitHub issue templates (#7361, #7396)
- Update installation guide to use newer macOS paths (#7311)
- Update macOS ESF documentation (#7303)
Packs
- Add Forcepoint Endpoint Chrome Extension detection to packs (#7346)
- Add
beurk
rootkit detection to packs (#7345)
Build
- Allow tests to reset the restarting state (#7373)
- Build librpm with ndb support (#7294)
- Customizable installation logic (#7315)
- Fix ASL test on macOS 11 and later (#7320)
- Restore query packs in Windows packaging (#7388)
- Skip deprecated ASL test when targeting macOS 10.13+ SDK (#7358)
- Update packaging commit to fix Linux symlinks (#7404)
- Update the CI Linux Docker image (#7332)
5.0.1
osquery 5.0 is a tremendously exciting release!
- We now install into /opt/osquery on macOS and Linux for better portability.
- Our default and recommended installation for macOS uses an application bundle to support entitlement-based features.
- We now use Endpoint Security APIs for various event-based tables on macOS (more to come in the future!)
- We now use an osquery-organization macOS code signing certificate.
There are several breaking changes:
- Installation paths have changes from
/usr/local
to/opt/osquery
on macOS and Linux (symlinks to executables are provided). - macOS codesigning is now done through the Osquery Foundation account.
- If you manage macOS full disk permission through a profile, you will need to update it.
See docs - We removed the deprecated
blacklist
key from the configuration (#7153) - Search semantics on the augeas table have changed to be more performant, but do break the existing query API.
Representing commits from 21 contributors! Thank you all.
Note: The linux .tar.gz
includes debugging symbols. This may be larger than you expect
Table Changes
- Add
secureboot
table for Linux and Windows (#7202) - Add
tpm_info
for Windows (#7107) - Fix
osquery_info
build_platform column value on Linux (#7254) - Support
pid_with_namespace
in more tables (#7132) - Update
augeas
table to use native pattern matching (BREAKING) (#6982) - Update
chrome_extensions
to include Edge & EdgeBeta (#7170) - Update
disk_encryption
table to support QueryContext (#7209) - Update
last
to include utmp type name column (#7201) - Update
sudoers
table to support newer include syntax (#7185) - Update
user_ssh_keys
to detect encryption of ed25519 keys (#7168)
Under the Hood Improvements
- Add ruby namespace to the thrift definition (#7191)
- Always initialize variable change in PerformanceChange (#7176)
- Remove deprecated
blacklist
key (#7153) - Use total_size within watchdog on Windows (#7157)
- Support AF_PACKET sockets reporting on Linux (#7282)
- socket_events improvements in Linux audit system (#7269)
Bug Fixes
- Add case sensitive pragma to the pragma/actions authorizer allow list (#7267)
- Add feature to skip denylist for event-based queries (#7158)
- Change logger_mode flag to be correctly interpreted as an octal (#7273)
- Do not let osquery create multiple copies of the extension running at once (#7178)
- Fix Linux audit rule removal upon osquery exit (#7221)
- Fix broadcasting empty logs to logger plugins (#7183)
- Fix issues applying ACLs during chocolatey deployment (#7166)
- Fix memory issue in Windows fileops (#7179)
- Fix
process_open_sockets
type error on darwin (#6546) - Make sure that the file action
MOVED_TO
is tracked with yara events. (#7203) - Prevent osquery from killing itself when the
--force
flag is used (#7295) - Prevent race condition between shutdown and worker or extension launch (#7204)
Documentation
- Add a security assurance case (#7048)
- Bring the YARA wiki page up to date (#7172)
- Spelling fixes (#7211, #7186)
- Update
uptime
table description (#7270) - Update osquery installed artifacts paths in the documentation (#7286)
Build
- Add TimeoutStopSec to systemd service files (#7190)
- Correct macOS installed app bundle path in osqueryctl and doc (#7289)
- Create an macOS app bundle (#7263)
- Fix choco packaging not failing when an error occurs during install or upgrade (#7182)
- Fix path in macOS launchd plist (#7288)
- Pin the packaging repo within GitHub workflows (#7208, #7255, #7279)
- Update Windows deployment icon to png (#7163)
- Update install paths, and remove deprecated Facebook naming (#7210)
- Update macOS build to include app bundle related files (#7184)
- Update osquery installed artifacts default paths in code (#7285)
- Update the installation path on Linux (#7271)
- libs: Add options to AWS Optionally enable debug option and restrict content-type header size for PUT req (#7216)
- libs: Enable and compile the YARA macho module on macOS (#7174)
- libs: Update OpenSSL to version 1.1.1l (#7293)
- libs: Update Strawberry Perl to 5.32.1.1, use HTTPS downloads (#7199)
- libs: Update ebpfpub (#7173, #7219)
5.0.0
Initial draft of the 5.0. This release may be deleted!
4.9.0
Representing commits from 16 contributors! Thank you all.
Note: The linux .tar.gz
includes debugging symbols. This may be larger than you expect
New Features
- Add filesystem logrotate feature (#7015)
- Add Non-Functional EndpointSecurity based process events to macOS (Requires updated codesigning due in 5.0) (#7046)
Table Changes
- Add
mdm_managed
column tosystem_extensions
on macOS (#6915) - Add
prefetch
table on Windows (#7076) - Add support for IMDSv2 to AWS tables (#7084)
- Enable container stats on docker containers that don't have traditional networks (#7145)
- Update
homebrew_packages
to include new prefix, and allow specifying alternate prefixes (#7117) - Update
ntfs_acl_permissions
to list all ACE entries (usingGetAce()
) (#7114) - Update
processes
table to display additional Windows attributes (secured
,protected
,virtual
,elevated
) (#7121) - Update how
package_install_history
identifies the packageIdentifiers key (#7099) - Update how
identifier
is calculated inchrome_extensions
(#7124)
Under the Hood improvements
- Improve speed of osquery shutdown procedure (#7077)
- Improve shutdown speed during initialization (#7106)
- Update website generators (#7136)
- CLI flag to allow osquery to keep retrying enrollment (instead of exiting) (#7125)
- rocksdb: Do not fsync WAL writes (#7094)
- Move CPack packaging to a dedicated repository (#7059)
- Restore thrift socket 5min timeout (#7072)
- Consolidate syscalls to a single audit rule (#7063)
Bug Fixes
- Add current WMI location for Dell BIOS info (#7103)
- Correct RocksDB error code and subcode printing on open failure (#7069)
- Fix
pipe_channel
not reading all data in a message (#7139) - Fix crash and deadlocks in recursive logging (#7127)
- Fix custom
curl_certificate
timeouts (#7151) - Fix extensions crash on shutdown (#7075)
- Handle updated paths on various macOS tables --
xprotect_entries
,xprotect_meta
,launchd
(#7138, #7154) - Trigger event cleanup checks every 256 events (#7143)
- Update generating an extension uuid to be thread safe (#7135)
- Watchdog should wait for the worker to shutdown (#7116)
Documentation
- Update process auditing requirements documentation (#7102)
- Update website docs indicating windows support for YARA tables (#7130)
- Add 4.9.0 CHANGELOG (#7152)
Build
- Add Apple provisioning profile for distribution (#7119)
- Add more tests for events expiration (#7071)
- CI: Regenerate sccache cache when compiler version changes (#7081)
- Fix flaky test test_daemon_sigint by waiting for pidfile (#7095)
- Fix icon in Windows packaging (#7148)
- Minor cleanup of unused variables (#7128)
- Print extension SDK minimum version required when failing to load (#7074)
- Remove POSIX-only
-fexceptions
flag on Windows (#7126) - Remove duplicated osquery_utils_aws_tests-test (#7078)
- Remove flaky test decorators for python tests (#7070)
- Update SQLite to version 3.35.5 (#7090)
- Update librdkafka to version 1.7.0 (#7134)
- Update libyara to version 4.1.1 (#7133)