ELF: Detect OS from Go binaries

[williballenthin]

use the strategies pioneered by GoReSym to detect the target OS for ELF binaries compiled by Go:

• find the GOOS configuration, and
• find embedded Go source filenames

closes #1978
FYI @C0d3R3ad3r
FYI @stevemk14ebr

Checklist

• No CHANGELOG update needed

• No new tests needed

• No documentation update needed

[mr-tz]

LGTM, my only comment would be on tests around this
if not part of this PR let's create an issue to track adding samples/tests

[mr-tz]

add a comment what these are?

[williballenthin]

thank you for the feedback - i was being lazy. i'll add some test files and some test cases.

[stevemk14ebr]

I'm not sure how reliable expecting the note to be present will be. Floss does this:

https://github.com/mandiant/flare-floss/blob/b2ca8adfc5edf278861dd6bff67d73da39683b46/floss/language/identify.py#L88

[stevemk14ebr]

I would recommend testing on old go versions prior to 1.18 when buildinfo was added. I would also recommend testing with binaries emitted by garble and gobfuscate which can mess with symbol names. Otherwise LGTM with the above notes in mind

[stevemk14ebr]

This is totally fine and should always work, but the buildinfo is just a string with some separator control words so could be easy to parse too.

Writing of buildinfo: https://github.com/golang/go/blob/1de46564a766f9647b22ebab0f35bccd14291460/src/runtime/debug/mod.go#L104

Reading of buildinfo, split by control words: https://github.com/mandiant/GoReSym/blob/0860a1b1b4f3495e9fb7e71eb4386bf3e0a7c500/runtime/debug/mod.go#L110-L118

[yelhamer]

LGTM!

[yelhamer]

The go version command uses logic similar to the one in the get_go_buildinfo_date() function (in addition to looking for the BUILDINFO_MAGIC) to check if an executable is go or not. See references to the errNotGoExe error here: https://github.com/golang/go/blob/master/src/debug/buildinfo/buildinfo.go#L41

Maybe consider incorporating that here? perhaps:

Suggested change

	if shdr.get_name(elf) == ".note.go.buildid":
	if shdr.get_name(elf) in (".note.go.buildid", ".go.buildinfo"):

Or maybe consider retrieving the buildinfo and testing whether the BUILDINFO_MAGIC is therein? perhaps after we check the section names?

Also, I think this might be a good place to make use of Google's Magika. Maybe if our heuristics can't determine the OS then output the error: "Input file does not appear to target a supported OS according to our heuristics. If you want to use Magika to guess the OS then run capa with the --use-magika parameter"?

[mr-tz]

Tests are TODO / in progress? Getting the following when trying to resolve conflicts:

Failed to merge submodule tests/data (commits don't follow merge-base)
CONFLICT (submodule): Merge conflict in tests/data
Recursive merging with submodules currently only supports trivial cases.
Please manually handle the merging of each conflicted submodule.
This can be accomplished with the following steps:
 - go to submodule (tests/data), and either merge commit 9f7f3c5
   or update to an existing commit which has merged those changes
 - come back to superproject and run:

      git add tests/data

   to record the above merge or update
 - resolve any other conflicts in the superproject
 - commit the resulting index in the superproject

Before continuing, I wanted to confirm what the status here was/is.

[williballenthin]

Yeah, I owe tests here. I believe the implementation is solid but without tests I can't prove it.

At a comfortable effort level, I could have this done in maybe two weeks. If there's a pending deadline, let me know and I can squeeze it in sooner.

[mr-tz]

This is not urgent at all! I just thought I may be able to lend a hand.