Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Capa v7.01 Not recognizing Linux x64 file #1978

Open
C0d3R3ad3r opened this issue Feb 8, 2024 · 10 comments · May be fixed by #1987
Open

Capa v7.01 Not recognizing Linux x64 file #1978

C0d3R3ad3r opened this issue Feb 8, 2024 · 10 comments · May be fixed by #1987

Comments

@C0d3R3ad3r
Copy link

C0d3R3ad3r commented Feb 8, 2024 •
edited by williballenthin

Description

Files being analyzed: docker-credential-pass-v0.8.0.linux-amd64 and linmux x64 file 23bae09b5699c2d5c4cb1b8aa908a3af898b00f88f06e021edcb16d7d558efad (both are linux x64 files)

Trying to analyze a linux x64 file and with no options set, just running ./capa [file], capa returns an error:

malware@malware-VirtualBox:~/Downloads$ ./capa -f auto docker-credential-pass-v0.8.0.linux-amd64 
ERROR:capa:--------------------------------------------------------------------------------
ERROR:capa: Input file does not appear to target a supported OS.
ERROR:capa: 
ERROR:capa: capa currently only supports analyzing executables for some operating systems (including Windows and Linux).
ERROR:capa:--------------------------------------------------------------------------------

However, when running with options explicitly set, ./capa -f auto --os linux [file], capa analyzes the file correctly.

malware@malware-VirtualBox:~/Downloads$ ./capa -f auto --os linux docker-credential-pass-v0.8.0.linux-amd64 
┍━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑                                                                        
│ md5                    │ 0e1ec55106a036ef12bee7220c221b5e                                                   │
│ sha1                   │ a18043386ff5f0ccfd63d79f534af8c599cf6969                                           │
│ sha256                 │ a69ce71a6b5cef7aadf343c93e00e1ffc549d649bd011fbb39bfa38534484511                   │
│ analysis               │ static                                                                             │
│ os                     │ linux                                                                              │
│ format                 │ elf                                                                                │
│ arch                   │ amd64                                                                              │
│ path                   │ /home/malware/Downloads/docker-credential-pass-v0.8.0.linux-amd64                  │
┕━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

┍━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑
│ ATT&CK Tactic          │ ATT&CK Technique                                                                   │
┝━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│ DEFENSE EVASION        │ Obfuscated Files or Information T1027                                              │
┕━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

┍━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑
│ MBC Objective               │ MBC Behavior                                                                  │
┝━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│ CRYPTOGRAPHY                │ Encrypt Data::AES [C0027.001]                                                 │
│                             │ Encrypt Data::RC4 [C0027.009]                                                 │
│                             │ Generate Pseudo-random Sequence::RC4 PRGA [C0021.004]                         │
├─────────────────────────────┼───────────────────────────────────────────────────────────────────────────────┤
│ DATA                        │ Check String [C0019]                                                          │
│                             │ Encode Data::Base64 [C0026.001]                                               │
│                             │ Non-Cryptographic Hash::FNV [C0030.005]                                       │
├─────────────────────────────┼───────────────────────────────────────────────────────────────────────────────┤
│ DEFENSE EVASION             │ Obfuscated Files or Information::Encoding-Standard Algorithm [E1027.m02]      │
│                             │ Obfuscated Files or Information::Encryption-Standard Algorithm [E1027.m05]    │
┕━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

┍━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┑
│ Capability                                            │ Namespace                                            │
┝━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┥
│ execute syscall instruction (20 matches)              │ anti-analysis                                        │
│ compiled with Go                                      │ compiler/go                                          │
│ encode data using Base64 (3 matches)                  │ data-manipulation/encoding/base64                    │
│ reference Base64 string                               │ data-manipulation/encoding/base64                    │
│ encrypt data using AES via x86 extensions (4 matches) │ data-manipulation/encryption/aes                     │
│ encrypt data using RC4 PRGA                           │ data-manipulation/encryption/rc4                     │
│ hash data using fnv                                   │ data-manipulation/hashing/fnv                        │
┕━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┙

malware@malware-VirtualBox:~/Downloads$ file 23bae09b5699c2d5c4cb1b8aa908a3af898b00f88f06e021edcb16d7d558efad 
23bae09b5699c2d5c4cb1b8aa908a3af898b00f88f06e021edcb16d7d558efad: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped

It looks like with this file (and a few other linux x64) files, capa fails to detect the file type and reports the error that the input file does not appear to target a supported OS

Steps to Reproduce

  1. run capa with no options passing the file through on the command line

Expected behavior:

Expected behavoir would be for capa to recognize the elf file and analyze it

Actual behavior:

capa errors with "Input file does not appear to target a supported OS."

Versions

capa v7.01
remnux 20.04

Additional Information

I am not sure if this is a bug that is not identifying these linux x64 files or not, but the workaround is to explicitly specify the --os linux on the command line which I would think should not be necessary (unless dealing with shellcode)
@williballenthin
Copy link
Collaborator

Hi @C0d3R3ad3r

Thanks for reporting this issue with so many details.

Guessing the underlying OS for an ELF file is tricky - there's not a header that says "this is for Linux". We have a bunch of heuristics that help us, but they don't work perfectly. You can see them here:

capa/capa/features/extractors/elf.py

Line 959 in fde1de3

def detect_elf_os(f) -> str:

23bae09b5699c2d5c4cb1b8aa908a3af898b00f88f06e021edcb16d7d558efad is a statically linked ELF file compiled by GCC. There aren't any hints in the ELF format about this targeting Linux, there's only a .note.gnu.property that specifies the CPU needs a few particular ISA features.

Within the file strings, there are some hints:

  • file paths, like /share/zoneinfo/ and /etc/localtime look like Linux paths, though BSD also uses /etc/localtime so its not very convincing.
  • the string LINUX_2.6

I will see if I can figure out if LINUX_2.6 is used by some common runtime and we can signature this, or if its derived from the malware's source code.

All in all, I think you did the right thing: to determine the sample targets Linux and provide the hint to capa. Unfortunately, capa won't always be able to recognize the underlying OS, so these hints are required.

@williballenthin
Copy link
Collaborator

the LINUX_2.6 string comes from a manual symbol resolution implementation related to VDSO routines (__vdso_clock_gettime). VDSO is describe here: https://man7.org/linux/man-pages/man7/vdso.7.html

We could inspect the strings of the file and note any of the symbol and version strings, using these to infer that VDSO on Linux is used. I don't really like that we're just guessing based on strings, but I suppose it's better than nothing.

@williballenthin
Copy link
Collaborator

a69ce71a6b5cef7aadf343c93e00e1ffc549d649bd011fbb39bfa38534484511 is a statically linked ELF file compiled from Go. There also aren't hints in the ELF format about this targeting Linux, but there is Go metadata that we could use.

Since Go is so prevalent, I think we should extend our OS detection code to recognize ELF binaries compiled by Go. The GoReSym technique is found here: https://github.com/mandiant/GoReSym/blob/0860a1b1b4f3495e9fb7e71eb4386bf3e0a7c500/main.go#L150-L178

@williballenthin
Copy link
Collaborator

williballenthin commented Feb 9, 2024 •
edited

Actually, here's the algorithm we should prefer to use: https://github.com/mandiant/GoReSym/blob/0860a1b1b4f3495e9fb7e71eb4386bf3e0a7c500/buildinfo/buildinfo.go#L105

https://github.com/mandiant/GoReSym/blob/0860a1b1b4f3495e9fb7e71eb4386bf3e0a7c500/buildinfo/buildinfo.go#L283-L295

@C0d3R3ad3r
Copy link
Author

Hopefully, these suggestions can be implemented in the next update to CAPA. For now, I can use the --os option to have capa analyze the files. Also, thank you for the quick response.

I wasn't sure how capa determined the file type. I would think something like the file command or custom file determination scheme would easily pick up that the file was a 64 bit elf file. In the meantime, I can script something up to use the file command and then pass the file along to capa with the proper --os options

@williballenthin
Copy link
Collaborator

A few minor comments here, not to be combative, but just for information:

would easily pick up that the file was a 64 bit elf file

We can tell the file is a 64-bit ELF (specifically, an ELF file with 64-bit little endian words), but this doesn't tell us the OS. While we can probably guess that most of these are for Linux, I'm afraid we'd lead too many people astray when encountering a FreeBSD file, for example. So, we try to be pretty sure, or we bail and ask the user to specify --os.

the file command

I went looking how how file/magic determines the OS, and it's surprisingly coarse. We already cover the logic that magic uses (to my knowledge), and have a few more heuristics. Though, if you find that magic does a better job, do share some details here and we can emulate it even better!

@williballenthin
Copy link
Collaborator

I have an implementation of OS detection for Go ELF files pending, should have that PR opened tomorrow. I think that will cover a lot of the edge cases that people see today. I'm glad that @C0d3R3ad3r mentioned it here.

@C0d3R3ad3r
Copy link
Author

Not taken as combative at all. Just trying to find the best way to process these types of files. I have encountered a lot of 64 bit elf files and show that they are not supported but using the --os option, capa runs as expected. I do appreciate all the comments and like I said, I am just trying to find some workaround for now. I get what you are saying though.

@williballenthin
Copy link
Collaborator

Feel free to post as many sample hashes as you like into this chat - I'd be happy to triage them and add heuristics as feasible.

@C0d3R3ad3r
Copy link
Author

As a quick fix, I wrote a python script which imports magic library so when I pass a file through to capa (so I can do this in bulk), I can compare strings looking for "ELF 64-bit" or "ELF 32-bit". It works for my purposes, but I hope in a later release of capa this can be automatic.

import magic
mime = magic.Magic(mime=False) // Setting to false outputs human readable ie. ELF 64-bit instead of application/x-executable

mimeType = mime.from_file(file_path)

    if (
        "ELF 64-bit" in mimeType
    ):
        format_args = ["--os", "linux"]

    if (
        "ELF 32-bit" in mimeType
    ):
        format_args = ["--os", "linux"]

'

Trivial, but for now it works.
Sample I tried was 443d71fbee450a75eadb258cc48289bc (upx unpack it)
Screenshot was just run from the command line

capa_32-bit-sample

@williballenthin williballenthin linked a pull request Feb 14, 2024 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ELF: Detect OS from Go binaries mandiant/capa
2 participants
@williballenthin @C0d3R3ad3r