Add application_execution tag to certain Amcache entries #4105

pyllyukko · 2022-05-06T08:48:06Z

One line description of pull request

Add application_execution tag to certain Amcache entries.

Description:

More application_execution tags. I compared this tagging to RegRipper's output and with the test data I was using they produced the same results.

Notes:

All contributions to Plaso undergo code
review. This makes sure
that the code has appropriate test coverage and conforms to the Plaso style
guide.

One of the maintainers will examine your code, and may request changes. Check off the items below in
order, and then a maintainer will review your code.

Checklist:

Automated checks (Travis, Codecov, Codefactor )pass
No new new dependencies are required or l2tdevtools has been updated
Reviewer assigned

joachimmetz · 2022-05-07T14:34:15Z

data/tag_windows.txt

@@ -11,6 +11,7 @@ application_execution
  data_type is 'windows:registry:mrulistex' AND entries contains '.exe'
  data_type is 'windows:registry:userassist' AND value_name contains '.exe'
  data_type is 'windows:tasks:job'
+  parser is 'winreg/amcache' AND data_type is 'windows:registry:key_value' AND values contains 'BundleManifestPath'


can you add a test for this as well in https://github.com/log2timeline/plaso/blob/main/tests/data/tag_windows.py

joachimmetz

please add missing test coverage

pyllyukko · 2022-05-10T09:11:47Z

please add missing test coverage

Yes. The test would probably look something like this:

from plaso.containers import windows_events

...

# Test: parser is 'winreg/amcache' AND
#       data_type is 'windows:registry:key_value' AND
#       values contains 'BundleManifestPath'
event = events.EventObject()
event.timestamp = self._TEST_TIMESTAMP
event.timestamp_desc = definitions.TIME_DESCRIPTION_MODIFICATION
event_data = windows_events.WindowsRegistryEventData()
event_data.values = 'BundleManifestPath: [REG_SZ]'
event_data.parser = 'winreg/amcache'
storage_writer = self._TagEvent(event, event_data, None)
self._CheckLabels(storage_writer, ['application_execution'])

Although I realized that I need to study this a little bit further. I'm trying to read this paper and am still somewhat confused in what cases AMCache has logged execution (which we're after) and when it's an installation.

There's probably also a better search term than BundleManifestPath. Something like looking at the key_path value etc.

(Win10) Reference: https://www.ssi.gouv.fr/uploads/2019/01/anssi-coriin_2019-analysis_amcache.pdf

codecov · 2022-06-12T05:19:06Z

Codecov Report

Merging #4105 (c902dbf) into main (daa4487) will decrease coverage by 0.02%.
The diff coverage is n/a.

❗ Current head c902dbf differs from pull request most recent head f372012. Consider uploading reports for the commit f372012 to get more accurate results

@@            Coverage Diff             @@
##             main    #4105      +/-   ##
==========================================
- Coverage   85.59%   85.56%   -0.03%     
==========================================
  Files         398      394       -4     
  Lines       33772    33488     -284     
==========================================
- Hits        28907    28655     -252     
+ Misses       4865     4833      -32

Impacted Files	Coverage Δ
plaso/single_process/extraction_engine.py	`78.60% <0.00%> (-1.09%)`	⬇️
plaso/output/formatting_helper.py	`84.64% <0.00%> (-0.77%)`	⬇️
plaso/output/rawpy.py	`93.54% <0.00%> (-0.21%)`	⬇️
plaso/cli/image_export_tool.py	`80.93% <0.00%> (-0.06%)`	⬇️
plaso/output/kml.py	`100.00% <0.00%> (ø)`
plaso/output/tln.py	`97.72% <0.00%> (ø)`
plaso/output/xlsx.py	`93.97% <0.00%> (ø)`
plaso/output/dynamic.py	`94.87% <0.00%> (ø)`
plaso/output/l2t_csv.py	`78.16% <0.00%> (ø)`
plaso/output/shared_dsv.py	`97.50% <0.00%> (ø)`
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update daa4487...f372012. Read the comment docs.

joachimmetz · 2022-06-12T05:21:47Z

tests/data/tag_windows.py

+    event_data.key_path = '\\Root\\InventoryApplicationFile\\7z.exe|afe683e0fa522625'
+    event_data.parser = 'winreg/amcache'
+    storage_writer = self._TagEvent(event, event_data, None)
+    self._CheckLabels(storage_writer, ['application_execution'])


Why not use _CheckTaggingRule ?

No real reason. I saw it written like this in other tests. Should I change it to _CheckTaggingRule?

yes, please, unless there is a specific reason not to, also see my comment about AMCacheFileEventData. The current filter rule is fragile since it tests a path as a string, not as a path using using PATH()

joachimmetz · 2022-06-12T10:08:24Z

Looking bit closer at this why not filter on AMCacheFileEventData ('windows:registry:amcache') instead of the generic 'windows:registry:key_value' ?

pyllyukko · 2022-06-13T16:30:45Z

Looking bit closer at this why not filter on AMCacheFileEventData ('windows:registry:amcache') instead of the generic 'windows:registry:key_value' ?

Because this particular artifact has that data type.

joachimmetz · 2022-06-13T17:10:53Z

Because this particular artifact has that data type.

Not really, it should create DefaultWindowsRegistryEvent on which you are creating the filter https://github.com/log2timeline/plaso/blob/main/plaso/parsers/winreg_plugins/amcache.py#L440 which represents the modification of the corresponding Windows Registry key and will also trigger on non-related keys with InventoryApplicationFile\\ in the path

And AMCacheFileEventData which you could base your filter on as well, which would only trigger on events generated from data under the InventoryApplicationFile key in an AMCache.hve file https://github.com/log2timeline/plaso/blob/main/plaso/parsers/winreg_plugins/amcache.py#L448

joachimmetz · 2022-06-13T17:33:52Z

Also none of the AMCacheFileEventData are "execution" events

plaso/plaso/parsers/winreg_plugins/amcache.py

Line 206 in 9172034

install_date_value = application_sub_key.GetValueByName('InstallDate')

From: https://www.ssi.gouv.fr/uploads/2019/01/anssi-coriin_2019-analysis_amcache.pdf

However, in InventoryApplicationFile, the last write
time of the keys always coincides with an execution of Microsoft Compatibility Appraiser. For a program
installation, the last write time of a key in Programs always coincides with an execution of ProgramDataUpdater,
while the last write time of a key in InventoryApplication coincides with the installation time of the program.

where Microsoft Compatibility Appraiser is presumably CompatTelRunner.exe

The key containing the information related to PEs is InventoryApplicationFile. It seems to list three categories
of PEs: executed shimmed EXE files with a GUI, EXE or SYS files that come with the installation of a program and
EXE files that are present in one of the directories scanned by Microsoft Compatibility Appraiser (Program
Files, Program Files x86 and Desktop). The execution of a PE appearing under this key can only be ascertained
if the PE is in the first category. For these, the last write time of the subkeys corresponds to the first execution date.
For the other PEs, the last write time of the subkeys is either the time of execution or the date of the first run of
Microsoft Compatibility Appraiser after the PE appeared, whichever comes first.
Hunting for illegitimate EXE or SYS files inside Program Files and Program File

so only for one of the categories executed shimmed EXE files with a GUI it seems to correspond to the first execution time according to this source

to me this reads like, reproducible tests would be quite beneficial here.

joachimmetz · 2022-09-11T06:36:12Z

@pyllyukko any updates?

pyllyukko · 2022-09-12T07:40:53Z

@pyllyukko any updates?

No updates. I should take the time to look deeper into this and provide some example data for testing. Sorry for leaving this hanging and I'll try to find the time in the near future to get this finalized.

joachimmetz · 2022-09-12T07:50:47Z

No updates. I should take the time to look deeper into this and provide some example data for testing.

Yes that would be great. I'll leave this PR open for now with the blocked and pending input tags.

pyllyukko · 2022-09-21T14:41:33Z

Also none of the AMCacheFileEventData are "execution" events

As I said earlier, the events this filter is after are not of that type. Here's an example event from the #3701 PR win10-Amcache.hve (maybe this could be a good candidate for reproducible test):

{
  "__container_type__": "event",
  "__type__": "AttributeContainer",
  "data_type": "windows:registry:key_value",
  "date_time": {
    "__class_name__": "Filetime",
    "__type__": "DateTimeValues",
    "timestamp": 132210024057296460
  },
  "display_name": "OS:/data/evidences/win10-Amcache.hve",
  "filename": "/data/evidences/win10-Amcache.hve",
  "inode": "-",
  "key_path": "\\Root\\InventoryApplicationFile\\csrss.exe|a9363ee544229f11",
  "message": "[\\Root\\InventoryApplicationFile\\csrss.exe|a9363ee544229f11] BinFileVersion: [REG_SZ] 10.0.18362.1 BinProductVersion: [REG_SZ] 10.0.18362.1 BinaryType: [REG_SZ] pe64_amd64 FileId: [REG_SZ] 00002038501676866b87cee4514ceff77daea9729f30 IsOsComponent: [REG_DWORD_LE] 1 IsPeFile: [REG_DWORD_LE] 1 Language: [REG_DWORD_LE] 1033 LinkDate: [REG_SZ] 04/26/2034 22:06:31 LongPathHash: [REG_SZ] csrss.exe|a9363ee544229f11 LowerCaseLongPath: [REG_SZ] c:\\windows\\system32\\csrss.exe Name: [REG_SZ] csrss.exe ProductName: [REG_SZ] microsoft® windows® operating system ProductVersion: [REG_SZ] 10.0.18362.1 ProgramId: [REG_SZ] 0000f519feec486de87ed73cb92d3cac802400000000 Publisher: [REG_SZ] microsoft corporation Size: [REG_QWORD] 17808 Usn: [REG_QWORD] 0 Version: [REG_SZ] 10.0.18362.1 (winbuild.160101.0800)",
  "parser": "winreg/amcache",
  "pathspec": {
    "__type__": "PathSpec",
    "location": "/data/evidences/win10-Amcache.hve",
    "type_indicator": "OS"
  },
  "timestamp": 1576528805729646,
  "timestamp_desc": "Content Modification Time",
  "values": "BinFileVersion: [REG_SZ] 10.0.18362.1 BinProductVersion: [REG_SZ] 10.0.18362.1 BinaryType: [REG_SZ] pe64_amd64 FileId: [REG_SZ] 00002038501676866b87cee4514ceff77daea9729f30 IsOsComponent: [REG_DWORD_LE] 1 IsPeFile: [REG_DWORD_LE] 1 Language: [REG_DWORD_LE] 1033 LinkDate: [REG_SZ] 04/26/2034 22:06:31 LongPathHash: [REG_SZ] csrss.exe|a9363ee544229f11 LowerCaseLongPath: [REG_SZ] c:\\windows\\system32\\csrss.exe Name: [REG_SZ] csrss.exe ProductName: [REG_SZ] microsoft® windows® operating system ProductVersion: [REG_SZ] 10.0.18362.1 ProgramId: [REG_SZ] 0000f519feec486de87ed73cb92d3cac802400000000 Publisher: [REG_SZ] microsoft corporation Size: [REG_QWORD] 17808 Usn: [REG_QWORD] 0 Version: [REG_SZ] 10.0.18362.1 (winbuild.160101.0800)"
}

So to my understanding this could be an execution event. Here's RegRipper output for the same data/event:

amcache v.20200515
(amcache) Parse AmCache.hve file

***InventoryApplicationFile***
c:\windows\system32\csrss.exe  LastWrite: 2019-12-16 20:40:05Z
Hash: 2038501676866b87cee4514ceff77daea9729f30

But I'm still confused about the paper's "three categories" and in which circumstances they happen and how we distinguish between those. Any thoughts how to handle this?

joachimmetz · 2022-09-22T04:27:09Z

But I'm still confused about the paper's "three categories" and in which circumstances they happen and how we distinguish between those.

that is why reproducible test data is so critical. This is not my top priority at the moment but I'll give it some thought when time permits.

joachimmetz reviewed May 7, 2022

View reviewed changes

joachimmetz requested changes May 7, 2022

View reviewed changes

joachimmetz self-assigned this May 7, 2022

pyllyukko marked this pull request as draft May 10, 2022 09:04

pyllyukko force-pushed the amcache branch from c902dbf to ceea80f Compare May 10, 2022 10:18

pyllyukko marked this pull request as ready for review May 10, 2022 10:19

pyllyukko force-pushed the amcache branch from ceea80f to 0a7b58c Compare May 10, 2022 10:21

pyllyukko requested a review from joachimmetz June 8, 2022 10:09

Add application_execution tag to certain Amcache entries

f372012

(Win10) Reference: https://www.ssi.gouv.fr/uploads/2019/01/anssi-coriin_2019-analysis_amcache.pdf

joachimmetz force-pushed the amcache branch from 0a7b58c to f372012 Compare June 12, 2022 05:18

joachimmetz reviewed Jun 12, 2022

View reviewed changes

joachimmetz added blocked Work cannot progress until another issue is resolved pending reporter input Issue is pending input from the reporter labels Sep 11, 2022

joachimmetz added needs closer look Issue that requires further analysis by a maintainer and removed pending reporter input Issue is pending input from the reporter labels Sep 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add application_execution tag to certain Amcache entries #4105

Add application_execution tag to certain Amcache entries #4105

pyllyukko commented May 6, 2022 •

edited

joachimmetz May 7, 2022 •

edited

joachimmetz left a comment

pyllyukko commented May 10, 2022

codecov bot commented Jun 12, 2022

joachimmetz Jun 12, 2022

pyllyukko Jun 12, 2022

joachimmetz Jun 12, 2022

joachimmetz commented Jun 12, 2022

pyllyukko commented Jun 13, 2022

joachimmetz commented Jun 13, 2022 •

edited

joachimmetz commented Jun 13, 2022 •

edited

joachimmetz commented Sep 11, 2022

pyllyukko commented Sep 12, 2022

joachimmetz commented Sep 12, 2022

pyllyukko commented Sep 21, 2022

joachimmetz commented Sep 22, 2022

Add application_execution tag to certain Amcache entries #4105

Are you sure you want to change the base?

Add application_execution tag to certain Amcache entries #4105

Conversation

pyllyukko commented May 6, 2022 • edited

One line description of pull request

Description:

Notes:

Checklist:

joachimmetz May 7, 2022 • edited

Choose a reason for hiding this comment

joachimmetz left a comment

Choose a reason for hiding this comment

pyllyukko commented May 10, 2022

codecov bot commented Jun 12, 2022

Codecov Report

joachimmetz Jun 12, 2022

Choose a reason for hiding this comment

pyllyukko Jun 12, 2022

Choose a reason for hiding this comment

joachimmetz Jun 12, 2022

Choose a reason for hiding this comment

joachimmetz commented Jun 12, 2022

pyllyukko commented Jun 13, 2022

joachimmetz commented Jun 13, 2022 • edited

joachimmetz commented Jun 13, 2022 • edited

joachimmetz commented Sep 11, 2022

pyllyukko commented Sep 12, 2022

joachimmetz commented Sep 12, 2022

pyllyukko commented Sep 21, 2022

joachimmetz commented Sep 22, 2022

pyllyukko commented May 6, 2022 •

edited

joachimmetz May 7, 2022 •

edited

joachimmetz commented Jun 13, 2022 •

edited

joachimmetz commented Jun 13, 2022 •

edited