Function identification

[williballenthin]

Some backends, such as IDA, already associate names with functions that they recognize. Others, such as viv, do not. It seems that capa rules could use this information, e.g. to match memcpy.

Some points of discussion:

• discrepancies among analysis backends and their naming
• signature format and efficacy
• rule syntax

[williballenthin]

IDA FLIRT

documentation: https://www.hex-rays.com/products/ida/tech/flirt/in_depth/

community signatures: https://github.com/Maktm/FLIRTDB

signature generator: https://github.com/fireeye/flare-ida/blob/master/python/flare/idb2pat.py

open source parser and matcher: https://github.com/williballenthin/lancelot/tree/master/flirt
open source parser and matcher (old): https://github.com/radareorg/radare2/blob/master/libr/anal/flirt.c

potential issue: its probably not ok to distribute FLIRT signatures, since Hex-Rays spends a lot of effort generating them to distribute to customers. so we should either rely on public data sets for our signatures or avoid relying on FLIRT as the only matching technology.

[williballenthin]

vivisect vamp

source: https://github.com/vivisect/vivisect/tree/master/vivisect/vamp

example sigs: https://github.com/vivisect/vivisect/blob/master/vivisect/vamp/msvc/__init__.py#L9

currently the signature format is relocation-masked exact byte match.
the documentation suggests these other (unimplemented) ideas:

• function arg count
• code block count
• globals refs
• code block refs
• unusual instruction use
• odd immediates
• import calls
• other signature calls
• certianty index
• Exception handling

[mr-tz]

I was able to get vamp running successfully pretty quickly with a local byte signature file.

Here's the difference for a random test binary:

$ diff -y --suppress-common-lines withsigs.txt regular.txt
functions: 1169                                               | functions: 1705
| query environment variable                           |      | | query environment variable (2 matches)               | host
| write file (2 matches)                               |      | | write file (3 matches)                               | host
| get disk information                                 |      | | get disk information (2 matches)                     | host
                                                              > | terminate process (2 matches)                        | host
                                                              > | terminate process via fastfail (3 matches)           | host
                                                              > | create thread                                        | host
| parse PE header (5 matches)                          |      | | link many functions at runtime                       | link
                                                              > | parse PE header (6 matches)                          | load

If we find a good solution to auto-generate signatures this is a promising approach.

For now, we could focus on the Visual Studio CRT (startup) functions which still may involve up to 15 library files * 10 Visual Studio versions * 2 architectures.

To further focus this we could first target functions that often generate FP hits such as parse PE header, write file, terminate process. For these cases capa-native matching may work equally well. The latter could have the benefit to abstract between the various Visual Studio versions and architectures.

[williballenthin]

capa-native matching

idea: use the features extracted by capa to match functions. this would be portable across backends.

for example:

• mnemonic count/distribution
• basic block count

potential issue: if one rule per signature, and there are 10s of thousands of signatures, that's a lot of rules, which may slow down analysis/matching.

[williballenthin]

Ghidra

source: https://github.com/NationalSecurityAgency/ghidra/blob/master/Ghidra/Features/FunctionID/src/main/java/ghidra/feature/fid/hash/MessageDigestFidHasher.java

community signatures: https://github.com/threatrack/ghidra-fidb-repo

In summary, its a cryptographic hash of masked instructions and code flow.

potential issue: it may be quite hard to mimic ghidra's logic on other backends.

[williballenthin]

PE exports

Trivial case: should use the name provided by the export table when present.