Update linkage_arm64.S

[MaddTheSane]

Add support for underscores in symbol names.

This is based off of my work on OpenEmu/Mupen64Plus-Core@decfb5d and OpenEmu/Mupen64Plus-Core@7c5a3eb

This is needed for Darwin/macOS and, I believe, Windows. (To get it to compile on Mac, you will need to use gas-preprocessor, but the current version doesn't like the adrp/add used by this file.)
To enable, pass -DLEADING_UNDERSCORE to the compiler/preprocessor/assembler.

[Narann]

Thanks, I suspect we need to update the Makefile too. Anyone with Apple system to test?

[richard42]

I have Apple systems, but no M1 or anything with an ARM CPU.

[MaddTheSane]

Even with these patches, I can't get Mupen64 running on M1 under OpenEmu.

[ZachBacon]

So I was pointed to this PR by @Jj0YzL5nvJ and I can say that it helps proceed further along the compilation process, however it's far from a finished PR based what I've tried to do.

So on my m1 mac, I ran make -C ./projects/unix NEW_DYNAREC=1 ARCH_DETECTED=64BITS CPU=ARM OS=OSX PLUGINDIR= SHAREDIR= BINDIR= MANDIR= LIBDIR= APPSDIR= ICONSDIR=icons all

Now I know this is rather behind current git head, but the ouput I got was this https://gist.github.com/ZachBacon/5c4d4997eb665ec840e080da6d829ce9

[Jj0YzL5nvJ]

Try with:

export CXXFLAGS='-stdlib=libc++'
export LDFLAGS='-mmacosx-version-min=10.7'
export OPTFLAGS='-O2'

And build with make -e.

And post your src/asm_defines/asm_defines_gas.h to discard gen_asm_script.sh malfunctions.

P.S: What assembler compiler are you using for ARM64?

[ZachBacon]

Try with:

export CXXFLAGS='-stdlib=libc++'
export LDFLAGS='-mmacosx-version-min=10.7'
export OPTFLAGS='-O2'

And build with make -e.

And post your src/asm_defines/asm_defines_gas.h to discard gen_asm_script.sh malfunctions.

P.S: What assembler compiler are you using for ARM64?

I'm using the default of whatever comes with the commandline tools within macOS for assembly

asm_defines_gas.h

[Jj0YzL5nvJ]

OSX + ARM64 is out of the scope... the current Makefile is useless for OSX on ARM64.

Replace:
AS = nasm for AS = as

Erase:

ifeq ($(ARCH_DETECTED), 64BITS)
  ASFLAGS = -f macho64 -d LEADING_UNDERSCORE
else
  ASFLAGS = -f macho -d LEADING_UNDERSCORE
endif

And add:

ifeq ($(CPU), ARM)
  ifeq ($(ARCH_DETECTED), 64BITS)
    CFLAGS += -pipe -arch arm64 -mmacosx-version-min=10.7 -isysroot $(OSX_SDK_PATH)
  endif
endif

[ZachBacon]

still no dice. Same error as I posted above for this branch and same error for git master that I posted in the issue's repo which was this.

error: unknown directive .hidden new_dyna_stop; .type new_dyna_stop, %function; new_dyna_stop:

[MaddTheSane]

Are you running the assembly file through gas-preprocessor?

[ZachBacon]

Are you running the assembly file through gas-preprocessor?

by gas, you mean as? Then yes.

[MaddTheSane]

gas-preprocessor, as in https://github.com/libav/gas-preprocessor/blob/master/gas-preprocessor.pl

[ZachBacon]

so basically you want me to adapt a step that's not in the makefile to bypass the issue... in order to get it linked with asm support.

[MaddTheSane]

Yep.

[Jj0YzL5nvJ]

https://wiki.libav.org/Tools/gas-preprocessor

# full use of the advanced options
gas-preprocessor.pl -arch aarch64 -as-type clang -- clang -arch arm64

[ZachBacon]

so at this point /if/ it's known to work with this pr, shouldn't it be integrated specifically for the MacOS m1 silicon chips? To have the makefile do the work instead of manual intervention?

[Jj0YzL5nvJ]

It will first be necessary to know the changes that lead to a successful compilation that works without errors...

According to OpenH264, NASM works in brew for ARM64...

[ZachBacon]

took me a little bit to get back to this due to work, and perhaps I am not understanding what is expected of me, and I do apologize for that. But I'm still being brought back to square one with this.

So what I did first was this make OS=OSX ARCH=arm64 ARCH_DETECTED=64BITS AS="./gas-preprocessor.pl -arch aarch64 -as-type clang -- clang -arch arm64" all which got me back to hitting my original issue of

../../src/device/r4300/new_dynarec/arm64/linkage_arm64.S:224:1: error: unknown directive .hidden new_dyna_stop; .type new_dyna_stop, %function; new_dyna_stop:

but then I thought of trying to build the file outside of the makefile by using that perl script that I was linked to. And I still got nothing but errors.

https://gist.github.com/ZachBacon/a8db9d70f8fdb1263aea99ed92731346

[Jj0YzL5nvJ]

Without a developer who implements the platform support, this will never be fixed.
Correct me if I'm wrong, but to me understand the Perl script is meant for asm_defines_gas.h.

Something like:

	perl $(SRCDIR)/../tools/gas-preprocessor.pl $(CC) $(OPTFLAGS) $(WARNFLAGS) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c -o $@ $<

Obviously you would have to add the script to tools folder for my example.

Try with:

export PIC=1
export OSX_SDK_PATH="$(shell xcrun --sdk macosx --show-sdk-path)"
export CFLAGS="-pipe -arch arm64 -DL_ENDIAN -fno-common -mmacosx-version-min=11.0 -isysroot $(OSX_SDK_PATH)"

[ZachBacon]

well, it's we are trying to figure this out, so that it can be implemented on this m1 platform, I'll give it a go and hope for the best.

[keegandent]

I have opened #908 which as-is works but with NO_ASM turned on. I will see if I can get these assembly changes integrated on my testbed M1 and if it still works when NO_ASM is turned off.

[keegandent]

Tried getting these changes up to date with master and messing with gas-preprocessor and the Makefile, but I'm still getting a an unknown-directive error. If anybody wants to take a look at my changes, I have them in https://github.com/keegandent/mupen64plus-core/tree/apple-silicon-asm

I have access to an M1 via MacInCloud if anyone without access to a testbed would like to schedule some collaboration time.

[MaddTheSane]

I, too, got unknown-directive errors. I just ended up manually fixing the end-result. I think it's a limitation of gas-preprocessor.

[keegandent]

I, too, got unknown-directive errors. I just ended up manually fixing the end-result. I think it's a limitation of gas-preprocessor.

What do you mean by “manually fixing”? Is there a way we can ship this manual adjustment or produce it programmatically? We could add that to the relevant section of the makefile if so.

[MaddTheSane]

I can't remember what I did to make gas-preprocessor to output the preprocessed assembly, but the closest I've got to a proper command line is gas-preprocessor.pl -arch aarch64 -as-type apple-clang -- clang -arch arm64 linkage_arm64.S -DLEADING_UNDERSCORE.

[MaddTheSane]

Ah, now I remember: the environment variable "GASPP_DEBUG", which writes the preprocessed source to STDOUT.

[keegandent]

I can't remember what I did to make gas-preprocessor to output the preprocessed assembly, but the closest I've got to a proper command line is gas-preprocessor.pl -arch aarch64 -as-type apple-clang -- clang -arch arm64 linkage_arm64.S -DLEADING_UNDERSCORE.

This is what I get out of that

user209549@TX233 unix % GASSP_DEBUG=1 ./gas-preprocessor.pl -verbose -arch aarch64 -as-type apple-clang -- clang -O2 -flto -Wall -fno-strict-aliasing -fvisibility=hidden -I../../src -I../../src/asm_defines -DM64P_PARALLEL -fPIC -pipe -arch arm64 -mmacosx-version-min=10.16 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -DLEADING_UNDERSCORE  -DNOCRYPT -DNOUNCRYPT -I../../subprojects/minizip  -I/Users/user209549/homebrew/Cellar/libpng/1.6.37/include/libpng16 -I/Users/user209549/homebrew/include/SDL2 -D_THREAD_SAFE -DM64P_OSD -I/Users/user209549/homebrew/opt/freetype/include/freetype2   -DNDEBUG -I../../subprojects/md5 -I../../subprojects/xxhash -DDYNAREC -DNEW_DYNAREC=4 -MD -MP ../../src/device/r4300/new_dynarec/arm64/linkage_arm64.S 
clang -O2 -flto -Wall -fno-strict-aliasing -fvisibility=hidden -I../../src -I../../src/asm_defines -DM64P_PARALLEL -fPIC -pipe -arch arm64 -mmacosx-version-min=10.16 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -DLEADING_UNDERSCORE -DNOCRYPT -DNOUNCRYPT -I../../subprojects/minizip -I/Users/user209549/homebrew/Cellar/libpng/1.6.37/include/libpng16 -I/Users/user209549/homebrew/include/SDL2 -D_THREAD_SAFE -DM64P_OSD -I/Users/user209549/homebrew/opt/freetype/include/freetype2 -DNDEBUG -I../../subprojects/md5 -I../../subprojects/xxhash -DDYNAREC -DNEW_DYNAREC=4 -MD -MP ../../src/device/r4300/new_dynarec/arm64/linkage_arm64.S -E
clang -O2 -flto -Wall -fno-strict-aliasing -fvisibility=hidden -I../../src -I../../src/asm_defines -DM64P_PARALLEL -fPIC -pipe -arch arm64 -mmacosx-version-min=10.16 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -DLEADING_UNDERSCORE -DNOCRYPT -DNOUNCRYPT -I../../subprojects/minizip -I/Users/user209549/homebrew/Cellar/libpng/1.6.37/include/libpng16 -I/Users/user209549/homebrew/include/SDL2 -D_THREAD_SAFE -DM64P_OSD -I/Users/user209549/homebrew/opt/freetype/include/freetype2 -DNDEBUG -I../../subprojects/md5 -I../../subprojects/xxhash -DDYNAREC -DNEW_DYNAREC=4 -MD -MP -x assembler -
<stdin>:308:5: error: unknown token in expression
 add, , @PAGEOFF
    ^
<stdin>:308:5: error: invalid operand
 add, , @PAGEOFF
    ^
<stdin>:313:5: error: unknown token in expression
 add, , @PAGEOFF
    ^
<stdin>:313:5: error: invalid operand
 add, , @PAGEOFF
    ^
user209549@TX233 unix % 

Let me know if there's anything else I should try. I'm beginning to wonder if we shouldn't benchmark performance with NO_ASM and see if these new M1 chips are good to go without further optimization. Or is it more than just acceleration and the emulator loses functionality without dynarec?

[Jj0YzL5nvJ]

Try with:
myzhang1029/gas-preprocessor
or
FFmpeg/gas-preprocessor

Libav variant is old...

[keegandent]

Try with: myzhang1029/gas-preprocessor or FFmpeg/gas-preprocessor

Libav variant is old...

Tried this, same issue. I'm editing my previous with a more verbose output.

[Jj0YzL5nvJ]

If we examine the libretro implementation, gas-preprocessor is only used in a single place...

[keegandent]

Tried with a ton of different compiler arguments from that libretro example, but no dice. Even for their iOS-64 implementation, it seems they go with NO_ASM. I think that's a pretty good indication this has a slim hope of working. I'll leave what I have in https://github.com/keegandent/mupen64plus-core/tree/apple-silicon-asm in case somebody else wants to take a crack at it.

[chrisd1100]

I got everything to compile fine (using the new dynarec) on an Apple M2 machine but it failed to run properly, generating continuous SIGILLs. One thought I had was the memory page size, I've seen this cause other emulator projects using mmap / dynamic recompiling to have issues. The default page size on Apple Silicon machines seems to be 16KB rather than the normal 4KB on most platforms. It also is 16KB on the default Raspbian Linux distro recommended for the Raspeberry Pi 5. I noticed hard coded 4096 and right shifts of 12 bits all over new_dynarec.c, and without knowing anything about how that code works, I'm wondering if this could be the issue.

[richard42]

No, I think this is just a coincidence. The new dynarec code uses 4k memory buffers in many places because the r4300 CPU which is being emulated uses 4k pages. But this has nothing to do with the page size of the underlying CPU running the emulator.