Futures: Fix SWIG compilation

[vpmedia]

clang version 3.2 (tags/RELEASE_32/final)
Target: avm2-unknown-freebsd8
Thread model: posix
"/cygdrive/f/crossbridge/sdk/usr/platform/cygwin/bin/clang++" -cc1 -triple avm2-unknown-freebsd8 -S
-disable-free -disable-llvm-verifier -main-file-name as3.cxx -mrelocation-model static -mdisable-fp
-elim -fmath-errno -momit-leaf-frame-pointer -v -g -coverage-file /tmp/as3-XJf3sY.s -resource-dir /c
ygdrive/f/crossbridge/sdk/usr/platform/cygwin/bin/../lib/clang/3.2 -dependency-file Modules/.deps/as
3.Tpo -sys-header-deps -MP -MT Modules/as3.o -D HAVE_CONFIG_H -D PCRE_STATIC -D __STDC_LIMIT_MACROS
-D __STDC_CONSTANT_MACROS -I ../Source/Include -I ../Source/CParse -I /cygdrive/f/crossbridge/swig-3
.0.0/Source/Include -I /cygdrive/f/crossbridge/swig-3.0.0/Source/DOH -I /cygdrive/f/crossbridge/swig
-3.0.0/Source/CParse -I /cygdrive/f/crossbridge/swig-3.0.0/Source/Preprocessor -I /cygdrive/f/crossb
ridge/swig-3.0.0/Source/Swig -I /cygdrive/f/crossbridge/swig-3.0.0/Source/Modules -I /cygdrive/f/cro
ssbridge/build/win/swig/pcre/pcre-swig-install/include -I /cygdrive/f/crossbridge/avm2_env/misc/ -I
/cygdrive/f/crossbridge/llvm-3.2/include -I /cygdrive/f/crossbridge/build/win/llvm-debug/include -I
/cygdrive/f/crossbridge/llvm-3.2/tools/clang/include -I /cygdrive/f/crossbridge/build/win/llvm-debug
/tools/clang/include -I /cygdrive/f/crossbridge/llvm-3.2/tools/clang/lib -isysroot /cygdrive/f/cross
bridge/sdk/usr/bin/../.. -fmodule-cache-path /var/tmp/clang-module-cache -Wno-long-long -fdeprecated
-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /cygdrive/f/crossbridge/build/win/swig/Sourc
e -ferror-limit 19 -fmessage-length 100 -mstackrealign -fno-rtti -fobjc-runtime=gcc -fcxx-exceptions
-fexceptions -fsjlj-exceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/as3-XJf3sY.s -
x c++ /cygdrive/f/crossbridge/swig-3.0.0/Source/Modules/as3.cxx
clang -cc1 version 3.2 based upon LLVM 3.2svn default target avm2-unknown-freebsd8

include "..." search starts here:

include <...> search starts here:

../Source/Include
../Source/CParse
/cygdrive/f/crossbridge/swig-3.0.0/Source/Include
/cygdrive/f/crossbridge/swig-3.0.0/Source/DOH
/cygdrive/f/crossbridge/swig-3.0.0/Source/CParse
/cygdrive/f/crossbridge/swig-3.0.0/Source/Preprocessor
/cygdrive/f/crossbridge/swig-3.0.0/Source/Swig
/cygdrive/f/crossbridge/swig-3.0.0/Source/Modules
/cygdrive/f/crossbridge/build/win/swig/pcre/pcre-swig-install/include
/cygdrive/f/crossbridge/avm2_env/misc
/cygdrive/f/crossbridge/llvm-3.2/include
/cygdrive/f/crossbridge/build/win/llvm-debug/include
/cygdrive/f/crossbridge/llvm-3.2/tools/clang/include
/cygdrive/f/crossbridge/build/win/llvm-debug/tools/clang/include
/cygdrive/f/crossbridge/llvm-3.2/tools/clang/lib
/cygdrive/f/crossbridge/sdk/usr/bin/../../usr/include/c++/v1
/cygdrive/f/crossbridge/sdk/usr/platform/cygwin/bin/../lib/clang/3.2/include
/cygdrive/f/crossbridge/sdk/usr/bin/../../usr/include
End of search list.
Stack dump:
0. Program arguments: /cygdrive/f/crossbridge/sdk/usr/platform/cygwin/bin/clang++ -cc1 -triple
avm2-unknown-freebsd8 -S -disable-free -disable-llvm-verifier -main-file-name as3.cxx -mrelocation-m
odel static -mdisable-fp-elim -fmath-errno -momit-leaf-frame-pointer -v -g -coverage-file /tmp/as3-X
Jf3sY.s -resource-dir /cygdrive/f/crossbridge/sdk/usr/platform/cygwin/bin/../lib/clang/3.2 -dependen
cy-file Modules/.deps/as3.Tpo -sys-header-deps -MP -MT Modules/as3.o -D HAVE_CONFIG_H -D PCRE_STATIC
-D __STDC_LIMIT_MACROS -D __STDC_CONSTANT_MACROS -I ../Source/Include -I ../Source/CParse -I /cygdr
ive/f/crossbridge/swig-3.0.0/Source/Include -I /cygdrive/f/crossbridge/swig-3.0.0/Source/DOH -I /cyg
drive/f/crossbridge/swig-3.0.0/Source/CParse -I /cygdrive/f/crossbridge/swig-3.0.0/Source/Preprocess
or -I /cygdrive/f/crossbridge/swig-3.0.0/Source/Swig -I /cygdrive/f/crossbridge/swig-3.0.0/Source/Mo
dules -I /cygdrive/f/crossbridge/build/win/swig/pcre/pcre-swig-install/include -I /cygdrive/f/crossb
ridge/avm2_env/misc/ -I /cygdrive/f/crossbridge/llvm-3.2/include -I /cygdrive/f/crossbridge/build/wi
n/llvm-debug/include -I /cygdrive/f/crossbridge/llvm-3.2/tools/clang/include -I /cygdrive/f/crossbri
dge/build/win/llvm-debug/tools/clang/include -I /cygdrive/f/crossbridge/llvm-3.2/tools/clang/lib -is
ysroot /cygdrive/f/crossbridge/sdk/usr/bin/../.. -fmodule-cache-path /var/tmp/clang-module-cache -Wn
o-long-long -fdeprecated-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /cygdrive/f/crossbri
dge/build/win/swig/Source -ferror-limit 19 -fmessage-length 100 -mstackrealign -fno-rtti -fobjc-runt
ime=gcc -fcxx-exceptions -fexceptions -fsjlj-exceptions -fdiagnostics-show-option -fcolor-diagnostic
s -o /tmp/as3-XJf3sY.s -x c++ /cygdrive/f/crossbridge/swig-3.0.0/Source/Modules/as3.cxx

1.  <eof> parser at end of file
   

2.  Code generation
   

3.  Running pass 'Function Pass Manager' on module '/cygdrive/f/crossbridge/swig-3.0.0/Source/Mo
   

dules/as3.cxx'.
4. Running pass 'AVM2 Assembly Printer' on function '@_ZN5clang19PreprocessorOptionsC2Ev'
clang++: error: unable to execute command: Aborted
clang++: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.2 (tags/RELEASE_32/final)
Target: avm2-unknown-freebsd8
Thread model: posix
clang++: note: diagnostic msg: PLEASE submit a bug report to and include the crash backtrace, prepr
ocessed source, and associated run script.
clang++: note: diagnostic msg:

---

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang++: note: diagnostic msg: /tmp/as3-pbefT3.cpp
clang++: note: diagnostic msg: /tmp/as3-pbefT3.sh
clang++: note: diagnostic msg:

https://dl.dropboxusercontent.com/u/1375050/cb_swig_log.zip

[vpmedia]

without -g (debug) flag it compiles but does not link (lot of member is not an ELF obj errors)

[mbolt35]

I've got some more information about this issue, but it pertains to adding back the -g flag. I think that -g is required for the specific clang and llvm api use inside as3.cxx

Note that I have not tested that this solution works, but I have tested to make sure that it will compile and link successfully.

Code Approach

The first approach I took was to try and update the CMemoryLayout to best match the clang tutorial found here: Clang Tutorial 1. I wasn't complete convinced that it would fix anything, but it seemed like a good first step. This didn't fix the issue, so I moved on to a hackier approach.

Hack Approach

When compiling the as3.cxx module, the compiler will start writing to a temporary .s file in the /tmp directory prefixed with as3-. When the compilation fails, that file is purged completely, thus the next step: sdk/usr/bin/avm2-as -o Modules/as3.o /tmp/as3-[file].s would fail because the .s file no longer existed.

In order to test adequately, I copied the breaking compile arguments so I could execute the compile directly. For my machine, it was this:

cd /cygdrive/c/cb/build/win/swig/Source && PATH=/cygdrive/c/cb/sdk/usr/bin:/usr/local/bin:/usr/bin:/cygdrive/c/Progra~1/Java/jdk1.6.0_43/bin CC=clang CXX=clang++ /cygdrive/c/cb/sdk/usr/bin/clang++ -cc1 -triple avm2-unknown-freebsd8 -S -disable-free -disable-llvm-verifier -main-file-name as3.cxx -mrelocation-model static -mdisable-fp-elim -fmath-errno -momit-leaf-frame-pointer -v -g -coverage-file /tmp/as3-jvXehn.s -resource-dir /cygdrive/c/cb/sdk/usr/bin/../lib/clang/3.2 -dependency-file Modules/.deps/as3.Tpo -sys-header-deps -MP -MT Modules/as3.o -D HAVE_CONFIG_H -D PCRE_STATIC -D __STDC_LIMIT_MACROS -D __STDC_CONSTANT_MACROS -I ../Source/Include -I ../Source/CParse -I /cygdrive/c/cb/swig-3.0.0/Source/Include -I /cygdrive/c/cb/swig-3.0.0/Source/DOH -I /cygdrive/c/cb/swig-3.0.0/Source/CParse -I /cygdrive/c/cb/swig-3.0.0/Source/Preprocessor -I /cygdrive/c/cb/swig-3.0.0/Source/Swig -I /cygdrive/c/cb/swig-3.0.0/Source/Modules -I /cygdrive/c/cb/build/win/swig/pcre/pcre-swig-install/include -I /cygdrive/c/cb/avm2_env/misc/ -I /cygdrive/c/cb/llvm-3.2/include -I /cygdrive/c/cb/build/win/llvm-debug/include -I /cygdrive/c/cb/llvm-3.2/tools/clang/include -I /cygdrive/c/cb/build/win/llvm-debug/tools/clang/include -I /cygdrive/c/cb/llvm-3.2/tools/clang/lib -isysroot /cygdrive/c/cb/sdk/usr/bin/../.. -fmodule-cache-path /var/tmp/clang-module-cache -Wno-long-long -fdeprecated-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /cygdrive/c/cb/build/win/swig/Source -ferror-limit 19 -fmessage-length 0 -mstackrealign -fno-rtti -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fsjlj-exceptions -fdiagnostics-show-option -o /tmp/as3-jvXehn.s -x c++ /cygdrive/c/cb/swig-3.0.0/Source/Modules/as3.cxx 

The idea is that I would be able to redirect the output into the .s into a permanent file so when the temporary file was deleted, I would have a copy of the last state it was in. In my compile options, the output was listed as /tmp/as3-jvXehn.s however, during execution, the file in /tmp is actually something like /tmp/as3-jvXehn-s[random characters]. So I opened a separate cygwin window, created a separate bash script which would tail -f -n 500000 an input file and output to as3-swig_out.s. In the first cygwin console, I ran the compile, when the temporary output was created in /tmp, I copied the file name and quickly pasted it into the parameter of my bash script which started tailing the file (and also prepending up to 500,000 lines before that). Once I see the failure in my compile, I can ctrl+c the bash script to end it, and note that it output to as3-swig_out.s

Because the above is somewhat difficult to explain, here's a short video of the process: Capturing the SWIG Compile Output

Once I had this output, I attempted to run the next step on it:

/cygdrive/c/cb/sdk/usr/bin/avm2-as -o as3.o as3-swig_out.s

This generated the following errors:

Error: Failed to execute compiler: D:\cygwin\tmp\alcaslQVlkzZo..as:42857
Syntax error: ')' is not allowed here
        i0 = (li32(ebp+))
                       ^

D:\cygwin\tmp\alcaslQVlkzZo..as:259053
Syntax error: Expected SCOPE_CLOSE but reached the end of the file instead.
} // function end
^


2 errors found

The line at the first error looks like this:

Ltmp6471:;
i1 = (li32(ebp-48))
i0 = (li32(ebp+)) // <== Error Here
si32(i1, ebp-200)
si32(i0, ebp-204)
i12 = (li32(ebp-60))

When I searched for uses of Ltmp6471 I could not find any, so I just added a 0 to get rid of the error: i0 = (li32(ebp+0))

The next error was a lot more obvious. The beginning of the package { had a leading { but since the generation failed prematurely, there wasn't a closing }. After line #259053 I added a closing } and re-ran avm2-as

With a successfully generated as3.o, I added this as $(BUILD)/swig/Source/Modules/as3.o, added a new Makefile build step:

swig-build2:    
    cd $(BUILD)/swig && PATH=$(SDK)/usr/bin:$(PATH) CC=$(FLASCC_CC) CXX=$(FLASCC_CXX) $(MAKE) && $(MAKE) install

Note that this new build step just omits the clean and cp from the build-swig step.

Then I ran make swig-build2 > build/win/logs/swig.txt 2>&1. This made it all the way until the final linking step and generated the error:

Modules/swigmain.o: error: undefined reference to '_swig_as3'

I returned back to the as3-swig_out.s file and searched for _swig_as3 which I found on line #70 without a global export. I added the global export from this:

;[GlobalMethod]
public function F_swig_as3():void {

to this:

;[GlobalMethod][Weak][Csym("W", "_swig_as3")]
public function F_swig_as3():void {

Then I used avm2-as again to rebuild as3.o and ran sdk/usr/bin/nm as3.o just to make sure that it exported the symbol correctly:

$ /cygdrive/c/cb/sdk/usr/bin/nm /cygdrive/c/cb/build/win/swig/Source/Modules/as3.o
00000000 W _swig_as3
00000000 W abort
00000000 W memcpy
00000000 W memmove
00000000 W memset

Then I replaced the broken as3.o with the new one and re-ran make swig-build2 > build/win/logs/swig.txt 2>&1.

This time, the swig build made it all the way to the end:

Installation complete
make[1]: Leaving directory '/cygdrive/c/cb/build/win/swig'

Aftermath

Like I said before, I have no idea if this is a solution at all. In fact, I'd probably guess that there's about a 90% chance it will fail. Mostly, because when the .s file dies, it still hasn't exported the // sections or // fixups which is responsible for registering the method calls (this is why _swig_as3 wasn't exported).

The real solution is fixing the SWIG code generation such that it actually compiles. This hack approach I took was simply to expose potential solutions. However, I'm out of ideas right now concerning the "correct" way to move forward. My experience with Clang, LLVM, and SWIG is limited.

File Attachments

All of the files I've referenced in this comment are zipped and located here: swig_as3_out.zip

• as3.cxx: Slightly modified Module file -- attempt to update CMemoryLayout
• as3-swig_out-BROKEN.s: The .s file up to the point of failure.
• as3-swig_out.s: The hacked up .s file which compiles
• as3.o: The compiled version of as3-swig_out.s using avm2-as
• collect_swig.sh: The bash script used to tail and capture the .s output.

[vpmedia]

Thanks for the post, I have a few ideas to try out:

• Enable debug logging / verbose error msgs for LLVM/Clang while leaving the temp files to exist after comp (Current verbose mode does not help at all, it should catch all sub-module console outputs or they should write to log file optionally).
• Create a simple unit test for the problem (Futures: Fix failing extralib - dmalloc #31):
• Try out LLVM 3.3 again