afl-gcc doesn't successfully create an instrumented file

[labba]

after using afl-gcc we get the flowing:

$ ./afl-gcc.exe -m32 -g my_test/copy_it.cpp -o my_test/copy_it.exe
afl-cc 1.96b by lcamtuf@google.com
[+] Executing:
gcc.exe -m32 -g my_test/copy_it.cpp -o my_test/copy_it.exe -B . -g -O3 -funroll-loops
afl-as 1.96b by lcamtuf@google.com
[+] Instrumented 84 locations (32-bit, non-hardened mode, ratio 100%).
/cygdrive/c/Users/user/AppData/Local/Temp/cceoHPlB.o:copy_it.cpp:(.text+0xf3a): undefined reference to native_init' /cygdrive/c/Users/user/AppData/Local/Temp/cceoHPlB.o:copy_it.cpp:(.text+0xf64): undefined reference tonative_shmat'
/cygdrive/c/Users/user/AppData/Local/Temp/cceoHPlB.o:copy_it.cpp:(.text+0xfb3): undefined reference to `native_fork'
collect2: error: ld returned 1 exit status

[labba]

i have added to compile with the winapi files and now it does generate an exe file:
i had to comment the "_S_IEXEC" in the winapi.c to make it work:
./afl-gcc.exe my_test/copy_it.cpp my_test/winapi.c my_test/winapi.h -m32 -g -o my_test/copy_it.exe

but now the file "EXE" that its generating is an invalid win32 application. when trying to load it into IDA for analysis IDA says:
"invalid file: bad ImageSize value FFC1A000. ..."

[labba]

also when compiling with the winapi,o file that was generated during the compilation of the:
WIN32=1 make

so when compiling like this:
$ ./afl-gcc.exe my_test/copy_it.cpp ./winapi.o -m32 -g -o my_test/copy_it.exe
afl-cc 1.96b by lcamtuf@google.com
[+] Executing:
gcc.exe my_test/copy_it.cpp ./winapi.o -g -o my_test/copy_it.exe -B . -g -O3 -funroll-loops
afl-as 1.96b by lcamtuf@google.com
[+] Instrumented 84 locations (32-bit, non-hardened mode, ratio 100%).

still getting bad win32 application with the same issue of image size.

[arizvisa]

When you ran make WIN32=1, did the build of test-instr.exe fail?

[labba]

yes (using win 7 x64 and cygwin x64):

here is a print of the output:
Makefile:187: warning: overriding recipe for target 'test_build'
Makefile:131: warning: ignoring old recipe for target 'test_build'
[] Checking for the ability to compile x86 code...
[+] Everything seems to be working, ready to compile.
[] Testing the CC wrapper and instrumentation output for Win32...
unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-gcc -c -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH="/usr/local/lib/afl" -DDOC_PATH="/usr/local/share/doc/afl" -DBIN_PATH="/usr/local/bin" -DVERSION="1.96b" -D_WIN32 -include winapi.h test-instr.c -o test-instr.o
unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-gcc winapi.o test-instr.o -o test-instr
#echo 0 | ./afl-showmap -m none -q -o .test-instr0 ./test-instr
#echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr
echo 0 | ./afl-showmap -m none -o .test-instr0 ./test-instr
afl-showmap 1.96b by lcamtuf@google.com
[*] Executing './test-instr'...

-- Program output begins --
-- Program output ends --

+++ Program killed by signal 11 +++

[-] PROGRAM ABORT : No instrumentation detected
Location : main(), afl-showmap.c:708

Makefile:187: recipe for target 'test_build' failed
make: *** [test_build] Error 1

[arizvisa]

Give me a moment to fully setup an win7 environment with cygwin x64 because I don't seem to encounter the same issue as you do right after cloning the afl-cygwin repo.

$ make WIN32=1
Makefile:187: warning: overriding recipe for target 'test_build'
Makefile:131: warning: ignoring old recipe for target 'test_build'
[*] Checking for the ability to compile x86 code...
[+] Everything seems to be working, ready to compile.
...
unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-gcc  winapi.o test-instr.o -o test-instr
#echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr
echo 0 | ./afl-showmap -m none -o .test-instr0 ./test-instr
afl-showmap 1.96b by <lcamtuf@google.com>
[*] Executing './test-instr'...

-- Program output begins --
Looks like a zero to me!
-- Program output ends --
[+] Captured 4 tuples in '.test-instr0'.
echo 1 | ./afl-showmap -m none -o .test-instr1 ./test-instr
afl-showmap 1.96b by <lcamtuf@google.com>
[*] Executing './test-instr'...

-- Program output begins --
A non-zero value? How quaint!
-- Program output ends --
[+] Captured 4 tuples in '.test-instr1'.
#@rm -f test-instr
/bin/sh: cmp: command not found
[+] All right, the instrumentation seems to be working!
[+] All done! Be sure to review README - it's pretty short and useful.
$ uname -a
CYGWIN_NT-6.1 MACHINE 2.4.1(0.293/5/3) 2016-01-24 11:26 x86_64 Cygwin
$ echo 0 | ./test-instr.exe
Looks like a zero to me!
$ echo 1 | ./test-instr.exe
A non-zero value? How quaint!

[labba]

what is your setup ? i can try to setup the same as you and see if that works
it's looking like you are on a win7 32bit ?
also what gcc you installed on cygwin? is it just gcc of is it ming?

[arizvisa]

Literally downloading the cygwin64 installer choosing the cygwin64 compilers nothing special.

$ uname -a
CYGWIN_NT-6.1 MACHINE 2.4.1(0.293/5/3) 2016-01-24 11:26 x86_64 Cygwin
$ echo 0 | ./test-instr.exe
Looks like a zero to me!
$ echo 1 | ./test-instr.exe
A non-zero value? How quaint!

But, give me a few hours for when I get off work and I'll be able to dig more into this for you.

[arizvisa]

Can you send me that 64-bit test-instr.exe that's being outputted? I wasn't able to reproduce it on 64-bit, but on 32-bit (after the 2nd re-install) I think I was able to reproduce what's happening on your system.

For some reason the data-directory entries in the executable emitted by afl-as are pointing to some really weird virtual addresses (import, export, and resource directory). I'm not sure how that's happening considering afl-as just looks for branch instructions by their label and injects a call to afl_maybe_log near them.

It shouldn't even be messing with any kind of section information or data-directories. Also because compilers don't really expose that type of information to the developer so I'm not sure how to fix it properly unless whatever it is that's being injected is causing some math done by cygwin's binutils to go wrong.

Essentially I want to see if the test-instr.exe on your system also has bad addresses for those 3 data-directory entries.

[arizvisa]

Also ftr, this seems to build fine in msys via gcc. So something that afl-as and cygwin's ld is doing is generating weirdly formatted PEs.

$ MSYS_NT-6.3 MACHINE 2.5.1(0.297/5/3) 2016-05-16 10:51 x86_64 Msys

[labba]

oops clicked the wrong button.
here is the file that was generated:
https://drive.google.com/file/d/0BzLLYqnUCc7yNVhLVmYwUWxJNWM/view?usp=sharing

i have made some tests and it's like that:
WinOS x64 and Cygwin64 - working
WinOS x86 and Cygwin x86 - not working
WinOS x64 and Cygwin x86 - not working

and i'm sorry but i had a WinOS7 x64 and Cygwin x86 and not a x64 like i have mentioned.

so it seems to be an issue for the Cygwin x86 version ..

[labba]

oops clicked the wrong button.
here is the file that was generated:
https://drive.google.com/file/d/0BzLLYqnUCc7yNVhLVmYwUWxJNWM/view?usp=sharing

i have made some tests and it's like that:
WinOS x64 and Cygwin64 - working
WinOS x86 and Cygwin x86 - not working
WinOS x64 and Cygwin x86 - not working

and i'm sorry but i had a WinOS7 x64 and Cygwin x86 and not a x64 like i have mentioned.

so it seems to be an issue for the Cygwin x86 version ..

so due to that we currently can only compile in 64-bit afl-gcc since the -m32 flag doesn't work in Cygwin64 (well at least i couldn't make it work so far)

[labba]

does this work on a win7 32-bit with Msys to successfully compile a afl-gcc of a 32-bit applications ?

[arizvisa]

It seems to for me.

[labba]

couldn't make the MingW compile the Makefile since missing sys/shm.h in MingW
what release have you used of MingW ?
i have found only this release to have the latest reqired "winternl.h" and etc: https://sourceforge.net/projects/mingw-w64/postdownload?source=dlp
but it lack the sys/shm.h ....

what have you used ?

[arizvisa]

There's a huge difference between msys and mingw. Your issues probably stem from that assumption. Keep in mind that altho you can compile things with mingw, most of the things that you're building are designed against a posix environment and thus you'll need one to use them.

• msys is the posix environment using the native windows libraries.
• cygwin is the posix environment but fully emulated on the windows platform.
• mingw is simply the gnu compilers ported to windows. (no posix).

You can find the latest Msys2 at https://msys2.github.io/.
Msys1 is at http://www.mingw.org/wiki/msys altho iirc it's not really maintained much.

On top of one of those, you'll need the w32api headers which contain the windows headers so that you can link with the native libraries under msys.

[labba]

ok i have used Msys2 and install all needed packages
OS: Win7-32-bit
$ uname -a
MSYS_NT-6.1 test-PC 2.5.1(0.297/5/3) 2016-05-16 10:13 i686 Msys

but still got bad instrumentation but exe seems to be valid this time:

unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-gcc  winapi.o test-instr.o -o test-instr
#echo 0 | ./afl-showmap -m none -q -o .test-instr0 ./test-instr
#echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr
echo 0 | ./afl-showmap -m none -o .test-instr0 ./test-instr
afl-showmap 1.96b by <lcamtuf@google.com>
[*] Executing './test-instr'...

-- Program output begins --
-- Program output ends --

+++ Program killed by signal 11 +++

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:708

make: *** [Makefile:192: test_build] Error 1

when trying to execute is it seems to work:

test@test-PC MSYS /afl-cygwin
$ echo 0 | ./test-instr.exe
Looks like a zero to me!

test@test-PC MSYS /afl-cygwin
$ echo 1 | ./test-instr.exe
A non-zero value? How quaint!

this has an effect on the execution of afl-fuzz because it always thinks that there is a crash ...
here is an example:

test@test-PC MSYS /afl-cygwin
$ ./afl-gcc.exe my_test/copy_it.cpp ./winapi.o -g -o my_test/copy_it.exe
afl-cc 1.96b by <lcamtuf@google.com>
[+] Executing:
gcc.exe my_test/copy_it.cpp ./winapi.o -g -o my_test/copy_it.exe -B . -g -O3 -funroll-loops
afl-as 1.96b by <lcamtuf@google.com>
[+] Instrumented 78 locations (32-bit, non-hardened mode, ratio 100%).

testing to run the application:

test@test-PC MSYS /afl-cygwin
$ ./my_test/copy_it.exe my_test/input/input.txt
Cpoy Len= 6 String: good

now trying to use it with afl-fuzz sometimes returns that there is a crash on the first test case and sometimes returns that there is no instrumentation.

[arizvisa]

k, it sounds like some race when it's trying to setup the initial shared memory page. i'll try and find time to look into this later today.

[labba]

strange.. if so i would have thought that when there is no issue in the race condition that the first test case will successfully pass and then continue to fuzzing.. but instead it crash just by loading the input sample..

[labba]

have you got into fixing it for 32bit yet ?

[arizvisa]

Sorry, no. I haven't gotten around to trying to reproduce your issue. Maintaining this project isn't my primary job or even my secondary job. Not only that but I don't even use AFL to fuzz things due it's key facility requiring source code in order to use it's instrumentation capability to measure fitness. This project was primarily a research project to demonstrate using an internal windows api to implement the fork system call on Windows for performance.

There is another project written by a sharp man (IvanFratic) called winafl that uses DynamoRIO for performing instrumentation and doesn't try to retain posix compatibility (and work together with msys/cygwin) as afl-cygwin does. This methodology is better if you actually intend to use AFL on the Windows platform to actually find vulnerabilities. A big benefit of using DynamoRIO is it not requiring source-code and so you shouldn't need to instrument at compile-time as you'd do with standard AFL. In the meantime, I'd recommend you take a look at that project as it appears from his github that he has less public projects to maintain.

Anyways, I just finished working on some OSX research related to exploiting a vulnerability targeting the magazine heap and after finishing that I'm now deep in another different project. When I'm finished with my current projects and I get free time I will work on reproducing your issue and updating this bug for you.

[labba]

thanks!