5.12.2

draft

5.12.1

Release notes coming soon

5.11.0

5.11.0

Git Commits

Representing commits from 11 contributors! Thank you all.

Table Changes

• Add new table vscode_extensions (#8150)
• Add support for additional Apple Silicon columns in secureboot table (#8215)
• Add Shortcut metadata parsing on Windows in the file table (#8143)
• Remove atom_packages table (#8181)
• Add additional chrome extensions paths (#8170) to pick up extensions for Chrome Beta, Chrome Dev, and Vivaldi.

Under the Hood improvements

• Add version collations to column definitions (#8222)
• Add support for additional collations in column definitions (#8214)
• Add version collate functions (#8168)
• Added cache and throttling for certificates, keychain_acls, and keychain_items tables (#8192). This is intended to reduce the occurrence of keychain corruption due to broken macOS APIs.
• process_open_sockets: Mark pid column as additional instead of index (#8191)

Bug Fixes

• Add stricter checks to JSON parsing (#8229)
• Fix signed/unsigned mismatch in powershell_events (#8225)
• Fix a crash in firefox_addons (#8227)
• Correct the aws_sts_region behavior (#8184)

Documentation

• Update building.md prereqs for Windows (#8216)
• Correct link to a PR in the 4.7.0 changelog (#8186)
• Call out in the CHANGELOG the format changes of the status logs decorations (#8174)
• Remove some duplicated lines from 5.8.1 changelog (#8172)
• Fix typo in table specs (#8163)
• Keychain cache and throttling documentation. (#8205)
• Changelog 5.10.2 (#8171)

Build / Dependencies

• Update libxml2 to v2.12.3 (#8223)
• Update zlib to 1.3 and ignore a CVE (#8218)
• Update openssl to 3.2.0 (#8212)
• Update nvdlib to use the latest NVD APIs (#8207)
• Fix Linux build (#8208)
• Correct job order (#8185)
• Re-enable tools_tests_testrelease (#8221)
• Enable client certificate verification in the TLS tests (#8211)
• Temporary workaround to build with XCode 15 (#8197)

5.10.2

5.10.2

Git Commits

This release has several updates and bugfixes. Several improvements to various tables, and their handling.

One potential breaking change, is in how the watchdog calculates CPU utilization.
Previously, this calculation was based on physical CPUs, now it is based on virtual cores. We believe this makes more sense with modern CPUs.

A second potential breaking change, is in PR #8102. In addition to allowing decorations to the top level of the status logs, this PR normalizes the decorations format to the results log. In practice, this means that the unixTime, severity and line JSON fields are now numbers instead of strings.

Representing commits from 18 contributors! Thank you all.

New Features

• Add --enable_watchdog_debug flag and improve watchdog error messages (#8070)
• Add --aws_enforce_fips to enforce AWS FIPS endpoints (#8075)
• Add new AWS valid regions (#8110)
• Implement decorations_top_level flag for status logs (#8102)

Table Changes

• Add new macOS SIP config flags (#8101)
• Added cloud_id to ycloud_instance_metadata - the vm metadata table for Yandex Cloud (#8086)
• Allow querying of kernel and filesystem drivers (#8119)
• Update es_process_file_events adding support for open events, and for only triggering on file_paths (#8114)
• Update firefox_addons to use rapidjson to parse and don't block on read (#8089)
• Update macOS es_process_events table: quote spaces in command line and environment variables (#8054)
• Update linux disk_encryption to recursively query parent crypt status (#8052)
• Add, and revert, indexing on block_devices (#8037, #8151)

Under the Hood improvements

• Add warnings when an enrollment secret cannot be found (#8082)
• Avoid blocking when reading plist files (#8099)
• Fix named virtual table create statement (#8139)
• Remove forensicReadFile (#8085)
• Substitute the TEXT macro with SQL_TEXT in table code (#8091)
• Use JSON member iterator instead of rescanning (#8122)
• core: Avoid checking if a file exists before opening (#8087)
• improvement: Avoid unnecessary string conversions (#8093)
• watchdog: Use virtual cores to calculate CPU utilization limit (#8104)

Bug Fixes

• Always lock event_index_mutex when accessing event_index map (#8077)
• Check audit return values with <= (#8125)
• Fix wifi_survey table not to crash if the ssid cannot be retrieved (#8153)
• Fix macOS EndpointSecurity FIM mute inversion for file paths (#8166)

Documentation

• Add a list of Osquery fleet managers (#7781)
• Add basic file carving documentation (#8118)
• Changelog for 5.9.1 (#8088)
• Changelog 5.10.1 (#8155)
• Fixed small doc error (#8147)
• Update Automatic Table Construction example (#8094)
• Update XCode version mentions to the proper one (#8128)
• Update the description of serial_number in connected_displays (#8113)

Build

• Fix openssl build arch for Windows ARM64 (#8134)
• Fix python test http server use SSLContext.wrap_socket() instead of deprecated ssl.wrap_socket() (#8169)
• GitHub Action to cleanup at stale ec2 runners (#8156)
• Ignore CVE-2023-30571 (#8065)
• Missing pragma/header guard for boottime.h (#8117)
• Permit cross compiling for x86_64 on Apple Silicon (#8136)
• build: update macos hosted github runner to macos-12 monterey (#8100)
• ci: Fix DistributedTests.test_run_queries_with_denylisted_query test (#8154)
• ci: Increase aarch64 available space by splitting the build (#8131)
• ci: Increase disk space on the Linux x86_64 runner (#8133)
• ci: Remove flakyness when removing unused packages on Linux (#8144)
• cve: Fix the expat product name in the libraries manifest (#8158)
• cve: Ignore dbus CVE-2023-34969 (#8126)
• cve: Ignore libcap CVE-2023-2603 (#8127)
• cve: Update expat to version 2.5.0 (#8159)
• cve: Update libmagic to 5.45 (#8142)
• cve: Update lzma to 5.4.4 (#8135)
• cve: Update openssl to 3.1.3 (#8141)
• libs: Fix openssl build on aarch64 (#8084)
• libs: Update openssl to 3.1.1 (#8081)
• libs: Update openssl to 3.1.2 (#8124)
• test: Fix leaks in inotify and rocksdb tests (#8080)

5.9.1

5.9.1

Git Commits

Big shoutout for the Windows Arm port!

Representing commits from 14 contributors! Thank you all.

New Features

• Add support for Windows on Arm (#7918)
• logger: Add new string_batch request type to compliment existing string type (#8027)

Table Changes

• Add connected_displays table on macOS (#7946)
• Add windows_search table (#7990)
• Restore functionality of crashes table on macOS 12 and newer (#7819)
• Update keychain_items to include data about key types (#8002)
• Update os_version to include Apple RSR fields using native API (#8011)
• Update safari_extensions to handle the current app extensions pattern (#7991)
• Update system_info to include the number of sockets (#8038)
• Update unified_log table to add predicate column and optimize timestamp constraint (#8019)

Under the Hood improvements

• Improving listDirectoriesInDirectory by using std::fs (#7974)
• Do not consider a 404 as an error in ec2-instance-metadata (#8025)
• Release objects and free memory obtained from COM (#7999)
• Do not pass wstring::c_str() to wstringToString function (#8000)
• Do not copy process arguments into vector for CreateProcess call (#7956)

Bug Fixes

• Fix version column in homebrew_packages (#8057)
• Improve extended_attributes implementation for Linux and macOS (#8046)
• Update event tables to mark time column as "additional" (#8020)

Documentation

• Update expired Slack invite (#8051)
• Update es_process_file_events.table description (#7978)
• CHANGELOG 5.8.2 (#7986)

Build

• cve: Update to openssl 1.1.1u (#8050)
• cmake: Add an option to disable shallow git clone operations (#8026)
• Fix the aarch64 workflow (#8036)
• test: Fix a leak in ExtendedAttributesTableTests SetUp function (#8045)
• cve: Update libxml2 to v2.11.2 (#8023)
• libs: Bring out LZ4 from rdkafka and update it to v1.9.4 (#7996)
• ci: Update python version and docs build tools (#7969)
• ci: Update aarch64 runner to Ubuntu 20.04 and update badges (#7984)
• Add few unit tests for the hashing component (#7993)

5.8.2

5.8.2

Git Commits

5.8.2 is a hotfix for how osquery's COM security initialization works See #7962 for details.

Representing commits from 6 contributors! Thank you all.

Bug Fixes

• Fix empty batch result set reporting (#7958)
• Fix COM security initialization by setting COM security per interface level (#7963)
• Fix username field in managed_policy table (#7944)

Documentation

• CHANGELOG 5.8.1 (#7957)

Build

• test: Do not always expect a row from the secureboot table (#7967)
• cmake: Only link against the experiments loader when needed (#7959)
• tests: Fix some tests becoming osquery shells (#7964)
• test: Fix SystemdUnitsTest missing the unit_file_state column (#7965)
• tests: Do not always build root tests on Linux (#7966)

5.8.1

5.8.1

Git Commits

Representing commits from 22 contributors! Thank you all.

New Features

• Record and send statistics for distributed queries (#7870)

Table Changes

• Add ETW-based process events table for Windows (#7821)
• Add pid_with_namespace for yara table (#7920)
• Add a new table kernel_keys to the Linux platform (#7876)
• Leave min_version empty in xprotect_meta when not specified (#7926)
• Port the secureboot table to macOS (#7692)
• Update docker_container_stats table to include cached_memory column (#7807)
• cpu_info: Port the table to macOS x86 and Apple Silicon (#7757)
• experiments: Implement a new bpf_process_events_v2 table (#7773)
• systemd_units: Add new unit_file_state column (#7895)

Under the Hood improvements

• Set counter consistently so zero always indicates all records (#7801)
• Support logging empty result set in batch format for initial runs (#7803)
• Support rollbacks of osquery when new versions introduce new column families (#7712)
• analysis.py: Add --pack flag to load queries from a pack file (#7935)
• profile.py: Log # of queries loaded and raise an error if 0 are loaded (#7934)

Bug Fixes

• Clear cached constraints and columns in xBestIndex (#7435)
• Fix assert fail for unverified WMI request result (#7921)
• Fix leaks in scheduled_tasks (#7903) (#7904)
• Flush console buffer during ungraceful exit (#7829)
• Propagate windows errors to the exit code (#7896)
• Relax osquery safe permissions check (#7763)
• Silence warnings for more builtin Chrome and Brave extensions (#7932)
• Workaround for hung routes table (#7916)
• dns_resolvers: fix typo in the name when spawning in namespace (#7875)
• test: Fix flaky test_daemon_sigint (#7888)

Documentation

• Add note about windows_security_products compatibility (#7880)
• CHANGELOG 5.7.0 (#7894)
• Docs: mention the recent adoption of automatic CVE scanning (#7878)
• Fix broken link in CODE_OF_CONDUCT.md (#7922)
• docs: Update the list of pages (#7866)
• docs: clarify that logger_plugin is set from CLI (#7917)

Build

• Do not catch table or registry exceptions when running tests (#7621)
• Fix and document discovery queries behavior on distributed queries and add tests (#7655)
• Try to free some disk space on the arm64 runners (#7950)
• ci: Automatically cancel old PR jobs (#7887)
• ci: Improve error message when a library is missing from the manifest (#7899)
• ci: Remove Windows 32bit build (#7939)
• ci: Update some actions to remove deprecation warnings (#7864)
• ci: Workaround in the aarch64 runner to avoid out of space (#7941)
• cmake: Remove forced static libraries search for osquery-toolchain (#7881)
• cve: Ignore libcryptsetup cves (#7871)
• cve: Ignore libdpkg CVE-2022-1664 (#7872)
• cve: Ignore libgcrypt cves (#7873)
• cve: Ignore sqlite CVE-2022-46908 (#7911)
• cve: Ignore util-linux cves (#7929)
• cve: Update librpm to 4.18.0 (#7910)
• cve: Update openssl to 1.1.1t (#7937)
• cve: Update yara to 4.2.3 (#7912)
• git: Ignore compile_commands.json and pyrightconfig.json (#7885)
• libs: Fix libmagic build on macOS (#7915)
• libs: Fix system paths used by dbus (#7919)
• libs: Update dbus to 1.12.24 (#7905)
• libs: Update libarchive to 3.6.2 (#7877)
• libs: Update libxml2 to 2.10.3 (#7882)
• libs: Update popt to 1.19 (#7909)
• libs: Update util-linux to 2.35.2 (#7902)
• libs: Update zlib to 1.2.13 (#7874)
• libs: update Thrift to 0.17 (#7868)
• test: Add an option to run only selected python testcases (#7890)
• test: Speed up ec2InstanceMetadata.test_sanity (#7907)

5.7.0

5.7.0

Git Commits

Representing commits from 12 contributors! Thank you all.

CVEs

Addressed by updating a library:

Ignored due to not affecting osquery:

• libzstd CVE-2021-24031 via (#7865)

New Features

• New table security_profile_info to retrieve security profile information on Windows (#7794)

Table Changes

• Add column to es_process_events for process codesigning flags (#7726)
• shimcache: Only check CurrentControlSet to avoid duplicate rows (#7832)
• processes: Fix the procfs memory unit kB, which is 1024 bytes not 1000 (#7818)
• Fix permissions on opening pipes for reading in pipes table (#7810)
• Fix the empty host column from logged_in_users table (#7685)
• docker_containers: Don't report finished_at for a container which is still running (#7783)
• processes: Stabilize the start_time column value on macOS and Linux (#7788)

Bug Fixes

• Do not access the AWS SDK request content type if missing (#7834)
• Fix deadlock when logging happens during a database reset (#7798)
• Fix handling of some errors during an AWS HTTP request (#7811)

Documentation

• CHANGELOG 5.6.0 (#7804)
• Add link to official YARA docs (#7792)
• Fix typo in keychain_items (#7790)

Packs

• packs/incident_response: process_memory_map is also applicable to Darwin (#7789)

Build

• cve: Ignore zstd CVE-2021-24031 (#7865)
• ci: Add a job and helper scripts to periodically scan for CVEs (#7787)
• ci: Update how we set github workflow step outputs (#7791)
• ci: Fix python version when installing modules and testing on macos (#7813)

5.6.0

5.6.0

Git Commits

Representing commits from 10 contributors! Thank you all.

Table Changes

• Add firmware_type column to platform_info on macOS (#7727)
• Add additional vendor support for the windows wmi_bios_info table (#7631)
• Fix docker_container_processes on macOS (#7746)
• Fix process_file_events subscriber being incorrectly initialized (#7759)
• Fix secureboot on windows by acquire the necessary process privileges (#7743)
• Improve macOS mdfind -- Reduce table overhead and support interruption (#7738)
• Remove binary column from firefox_addons table (#7735)
• Remove is_running column from macOS running_apps table (#7774)

Under the Hood improvements

• Add notes field to the schema and associated json (#7747)
• Add extended platforms to the schema and associated json (#7760)
• Fix a leak and improve users and groups APIs on Windows (#7755)
• Have --tls_dump output body to stderr (#7715)
• Improvements to osquery AWS logic (#7714)
• Remove leftover FreeBSD related code and documentation (#7739)

Documentation

• CHANGELOG 5.5.1 (#7737)
• Correct the description on how to configure and use Yara signature urls (#7769)
• Document difference between yara and yara_events (#7744)
• Link to the slack archives (#7786)
• Update docs: _changes tables are not evented (#7762)

Build

• Delete temporary CTest files (#7782)
• Fix table tests for macOS running_apps (#7775)
• Fix table tests for windows platform_info (#7742)
• Migrate jobs from ubuntu-18.04 to ubuntu-20.04 (#7745)
• Remove unused find_packages modules and submodule (#7771)

5.5.1

Git Commits

Osquery 5.5.1 has some really exciting table updates! There is a much anticipated unified_log for macOS, this table is the replacement for asl, and uses the current Apple APIs. Additionally, several tables have improved their cross-platform support.

Representing commits from 14 contributors! Thank you all.

New Features

• Add denylist mechanism to distributed queries (#7675)

Table Changes

• Add cgroup_path column to processes table on Linux (#7728)
• Add firmware_type column to platform_info table on Windows. (#7710)
• Add unified_log table for macOS (UAL) (#7598, #7713)
• Port memory_devices table to Windows (#7633)
• Port platform_info table to M1 Macs (#7660)
• Restore macOS kernel_panics table on modern macOS (#7585)
• Update battery table on macOS m1 with correct raw battery max and current capacity (#7721)
• Update mdfind query timeout to 30 seconds (#7725)
• Update macos password_policy table to use use -1 as sentinel value for uid column (#7699)
• Update parsing of authorized_keys file (#7560)
• Update the registry table to be case insensitive for key (#7708)

Under the Hood improvements

• Add a mechanism to reduce memory retained on Linux (#7502)
• Add denylist mechanism to distributed queries (#7675)
• Add table spec support for COLLATE NOCASE (#7680)
• Improve Pidfile handling (#7304)
• Prevent the audit event system from using too much memory (#7329)
• carves: use full pathnames while creating an archive (#7681)

Bug Fixes

• Fix GetMemorySize for Windows memory_devices table (#7711)
• Fix tpm_info bug where values were out of date (#7686)
• Fix a crash when parsing ATC config with no columns (#7693)
• Fix bug in GetHomeDirectories filesystem function (#7705)

Documentation

• Add core to the type column description of osquery_extensions schema (#7716)
• Add documentation about 3rd-party dependency security (#7684)
• Add example for hostname form in curl_certificate table (#7706)
• Adds info on how to use GTEST_FILTER on windows (#7696)
• Changelog 5.4.0 (#7678)
• Describe user-context-related caveat for screenlock table (#7649)
• Update schema for process_open_sockets.state (#7733)
• Update schema to reflect platform_info columns not available in Windows (#7732)

Build

• Add validation integration test for memory_devices (#7722)
• Temporarily disable memory_devices integration test (#7717)
• Update minimum macOS support from 10.12 to 10.14 (#7707)
• ci: Update and temporarily disable the macOS Catalina test job (#7700)
• cmake: Prevent defining some Linux only targets on other platforms (#7672)
• libs: Update libxml2 to v2.9.14 (#7729)
• libs: Update sqlite to version 3.39.2 (#7736)
• test: Fix Mdfind.test_sanity flakyness (#7701)