[emmake]compile OpenBLAS using emmake

[924657644]

I tried to compile OpenBLAS using emmake, but I had the following problems:

Is there any other way I can solve the problem? I want to compile the.c file that contains the OpenBLas function as the.wasm file

[brada4]

You have to use GNU make, or alternatively cmake e.g.
emmake make TARGET=GENERIC
or so.
What is your CPU? Is something close in TargetList.txt ?
"not supported" message says there is no specific support there, you can still use GENERIC target that consists of pure C files.

[martin-frbg]

What is your cpu architecture ? The error comes about because your compiler apparently does not define one of the usual architecture macros like __x86_64__ or __aarch64__ . I suspect that is a feature of emscripten (if that is what your emmake belongs to) as it is supposed to compile to "web assembly", and it will probably require some porting effort to make OpenBLAS compile in this environment. Most likely, none of the cpu-specific assembly code will be translatable, so your best bet is to compile for TARGET=GENERIC.
To get around this first problem, add #define OPENBLAS_SUPPORTED somewhere before line 1687 of getarch.c - but the "missing" functions get_libname and get_cpuconfig are implemented in the architecture-specific cpuid_*.c sources - with no recognized architecture, you need to supply your own (e.g. have get_libname just printf("wasm"); and get_cpuconfig printf("#define GENERIC\n#define DTB_DEFAULT_ENTRIES 64\n") and see how far that gets you.

[martin-frbg]

From https://github.com/nemequ/pre-defined-macros/blob/master/c/emcc/Linux-x86_64-1.37.9.h the Emscripten emcc appears to define __asmjs__, so you could make that addition in getarch.c something like:

#ifdef __asmjs__
#include "cpuid_asmjs.c"
#define OPENBLAS_SUPPORTED
#endif

next copy cpuid_sparc.c to cpuid_asmjs.c and edit the new file:

1. make get_architecture print X86_64 instead of SPARC
2. make get_subarchitecture print UNKNOWN
3. make get_subdirname print x86_64
4. change the "SPARC" in get_cpuconfig to "GENERIC"
5. make get_libname print asmjs (or wasm - as you prefer, this is what ends up in the libopenblas_xxx library name)
6. make get_corename print UNKNOWN

[brada4]

And probably ONLY_CBLAS=1 to omit fortran prototypes altogether.

[martin-frbg]

So I had another quick look at this - the initial messages about getarch.c are misleading, as this is basically a cross-compilation anyway, so no need to actually autodetect platform details and in particular no need to compile getarch itself to webasm. Also,
the current emcc does not appear to define __asmjs__ anymore (it has __wasm__ and __EMSCRIPTEN__).

A quick hack would be to add the following lines at the end of ctest.c

#if defined(__EMSCRIPTEN__)
ARCH_RISCV64
OS_WINDOWS
#endif

then in file common.h, around line 387 add this:

#ifdef __EMSCRIPTEN__
#define YIELDING
#endif

and in Makefile.rule, uncomment the line that has COMMON_OPT=-O2 and add -Wno-implicit-function-declaration

With these changes, make CC=emcc HOSTCC=gcc TARGET=RISCV64_GENERIC NOFORTRAN=1 USE_THREAD=0
produces a libopenblas that identifies as "WebAssembly (wasm) binary module version 0x1 (MVP)" but I have made no attempt to check if it actually works in a wasm context.
(The choice of RISCV64 for the GENERIC build has no consequence for the resulting code, except that it avoids complications
from inline assembly code that an x86_64 build would bring in, both use the same plain C sources. Similarly the choice of Windows as the OS is only to avoid Linux-specific headers and the syscalls they contain. As the USE_THREAD=0 signifies,
this is a single-threaded build - multithreading would bring in additional complications like signal handling that would need to be ported.)

[martin-frbg]

@924657644 any chance you could test this, or at least suggest a test for the generated webassembly file ? Otherwise I intend to close this. (I noticed there is also https://github.com/likr/emlapack , an emscripten port of an old version of the unoptimized reference BLAS/LAPACK implementation)

[ogrisel]

I tried to build openblas with emscripten as @martin-frbg proposed above, but then I got the following crash when generating the final .so file:

Traceback (most recent call last):
  File "/io/code/emsdk-linux/upstream/emscripten/emcc.py", line 4147, in <module>
    sys.exit(main(sys.argv))
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/io/code/emsdk-linux/upstream/emscripten/emcc.py", line 4140, in main
    ret = run(args)
  File "/io/code/emsdk-linux/upstream/emscripten/emcc.py", line 1208, in run
    phase_calculate_system_libraries(state, linker_arguments, linker_inputs, newargs)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/io/code/emsdk-linux/upstream/emscripten/emcc.py", line 2862, in phase_calculate_system_libraries
    extra_files_to_link += system_libs.calculate(all_linker_inputs, newargs, forced=state.forced_stdlibs)
  File "/io/code/emsdk-linux/upstream/emscripten/tools/system_libs.py", line 1958, in calculate
    handle_reverse_deps(input_files)
  File "/io/code/emsdk-linux/upstream/emscripten/tools/system_libs.py", line 1792, in handle_reverse_deps
    symbolses = building.llvm_nm_multiple([os.path.abspath(t) for t in input_files])
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/io/code/emsdk-linux/upstream/emscripten/tools/building.py", line 185, in llvm_nm_multiple
    for key, value in parse_llvm_nm_symbols(results.stdout).items():
  File "/io/code/emsdk-linux/upstream/emscripten/tools/building.py", line 581, in parse_llvm_nm_symbols
    status = line[entry_pos + 11] # Skip address, which is always fixed-length 8 chars.
IndexError: string index out of range

I edited /io/code/emsdk-linux/upstream/emscripten/tools/building.py to print the value of line, entry_pos and len(line) and here is the stdout just before the crash:

[...]
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0012c5be T sgtsvx_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0012c896 T sgttrf_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0012cc5d T sgttrs_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0012ce59 T sgtts2_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0012d4cd T shgeqz_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 00130906 T shsein_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 00131134 T shseqr_
60
80
/io/code/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0000082a t signature_mismatch:caxpy_
91
98

so the signature_mismatch:caxpy_ symbol is making emcc.py crash.

[ogrisel]

If I replace the .rfind by .find in the parse_llvm_nm_symbols function I can build but I am not sure the result is correct. I might will try to see if the generated .so file works with pyodide tomorrow.

[martin-frbg]

This may have been broken by some commits in the meantime, or you may be using a different version of emscripten ? Not sure where the "signature mismatch" for caxpy comes from, certainly don't remember getting it in my test build back in september.

[lesteve]

@ogrisel I had a similar error with emscripten 3.1.24 but somehow it seems to work without error in 3.1.27 (latest released version).

For clarity, here is how I am building for wasm (copying an pasting the relevant instructions from #3640 (comment)):

• in Makefile.rule, uncomment the line that has COMMON_OPT=-O2 and add -Wno-implicit-function-declaration

diff --git a/Makefile.rule b/Makefile.rule
index 5e6cefc2..c18ea288 100644
--- a/Makefile.rule
+++ b/Makefile.rule
@@ -231,7 +231,7 @@ NO_AFFINITY = 1
 # Common Optimization Flag;
 # The default -O2 is enough.
 # Flags for POWER8 are defined in Makefile.power. Don't modify COMMON_OPT
-# COMMON_OPT = -O2
+COMMON_OPT = -O2 -Wno-implicit-function-declaration
 
 # gfortran option for LAPACK to improve thread-safety
 # It is enabled by default in Makefile.system for gfortran

• run make CC=emcc HOSTCC=gcc TARGET=RISCV64_GENERIC NOFORTRAN=1 USE_THREAD=0

For completeness sake, I was not getting a traceback with 3.1.24 but an error saying

emcc: error: error parsing output of llvm-nm: `/tmp/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0000082a t signature_mismatch:caxpy_`

Excerpt from the make output:

emcc -O2 -Wno-implicit-function-declaration -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DC_LAPACK -DNO_WARMUP -DMAX_CPU_NUMBER=4 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.21.dev\" -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I..  -w -o linktest linktest.c ../libopenblas_riscv64_generic-r0.3.21.dev.so  && echo OK.
emcc: error: error parsing output of llvm-nm: `/tmp/OpenBLAS/libopenblas_riscv64_generic-r0.3.21.dev.so: 0000082a t signature_mismatch:caxpy_`
If the symbol name here contains a colon, and starts with __invoke_, then the object file was likely built with an old version of llvm (please rebuild it).
make[1]: *** [Makefile:224: ../libopenblas_riscv64_generic-r0.3.21.dev.so] Error 1
make[1]: Leaving directory '/tmp/OpenBLAS/exports'
make: *** [Makefile:132: shared] Error 2

Maybe (close to random guess) this is the fix from emscripten-core/emscripten#18152.

[ogrisel]

So supposedly the emcc fix was merged as emscripten-core/emscripten@cba63c7 so included in 3.1.27 3.1.26 3.1.25 but my emcc version is: 3.1.21 although I installed it with the emsdk install latest command...

[lesteve]

Not a emsdk expert, but what I do is something like:

# need to git clone only the first time
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk

./emsdk install latest
./emsdk activate latest
source emsdk_env.sh

❯ which emcc
/home/lesteve/dev/emsdk/upstream/emscripten/emcc

❯ emcc --version
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.27 (afa75f342eef4d925172479afa8e6233eb0ae5a9)
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[ogrisel]

I am pretty sure I did exactly the same...

[martin-frbg]

Well, the version of emscripten I tried it with has emcc -v returning 3.1.20. No error for make CC=emcc HOSTCC=gcc TARGET=RISCV64_GENERIC NO_FORTRAN=1 NO_LAPACK=1.
Trying to include LAPACK ends with lots of signature mismatches between LAPACK and LAPACKE however - mostly one knowing a function as void and the other as int. I have not yet checked if this is a bug in the f2c-created C LAPACK or sloppyness in the netlib LAPACKE.

[martin-frbg]

That's f2c converting all SUBROUTINEs to functions that return int. I'm preparing a PR

[lesteve]

I am seeing some warnings like this when building OpenBLAS, is that related to what you are saying?

wasm-ld: warning: function signature mismatch: daxpy_
>>> defined as (i32, i32, i32, i32, i32, i32) -> i32 in ../libopenblas_riscv64_generic-r0.3.21.dev.a(dgbcon.o)
>>> defined as (i32, i32, i32, i32, i32, i32) -> void in ../libopenblas_riscv64_generic-r0.3.21.dev.a(daxpy.o)

If that's the case that would be very nice, because these warnings turn into errors when I am trying to link scipy to OpenBLAS in Pyodide, see pyodide/pyodide#3331 (comment)

[martin-frbg]

Yes, that is exactly it. When I tested the compilation with emscripten back in september, I only tried with the BLAS parts of OpenBLAS and not the LAPACK.

[martin-frbg]

This is unfortunately taking a lot longer than I had anticipated - I hope to have the PR ready sometime tomorrow.

[lesteve]

OK sounds great, thanks a lot!

[lesteve]

@martin-frbg thanks a lot for your PR #3861, I still see some signature mismatch warnings when compiling the same way as #3861 (comment)?

Maybe these warnings are not that important? Some of them have test in their name, so I assume they are not crucial, but some of them don't have test in their name, for example:

emcc -O2 -Wno-implicit-function-declaration -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DC_LAPACK -DNO_WARMUP -DMAX_CPU_NUMBER=4 -DMAX_PARALLEL_N
UMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.21.dev\" -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCH
AR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I.. -DADD_ -DCBLAS -o xdcblat2 c_dblat2c.o c_dblas2.o c
_d2chke.o auxiliary.o c_xerbla.o constant.o ../libopenblas_riscv64_generic-r0.3.21.dev.a -L/home/lesteve/dev/emsdk/upstream/emscripten/cache/sysroot/lib/wasm
32-emscripten  -lGL -lal -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lc++-noexcept -lc++abi-debug-noexcept -lsockets  
emcc -O2 -Wno-implicit-function-declaration -DMAX_STACK_ALLOC=2048 -Wall -DF_INTERFACE_GFORT -fPIC -DC_LAPACK -DNO_WARMUP -DMAX_CPU_NUMBER=4 -DMAX_PARALLEL_N
UMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.21.dev\" -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCH
AR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I.. -DADD_ -DCBLAS -o xccblat2 c_cblat2c.o c_cblas2.o c
_c2chke.o auxiliary.o c_xerbla.o constant.o ../libopenblas_riscv64_generic-r0.3.21.dev.a -L/home/lesteve/dev/emsdk/upstream/emscripten/cache/sysroot/lib/wasm
32-emscripten  -lGL -lal -lstubs-debug -lnoexit -lc-debug -ldlmalloc -lc++-noexcept -lc++abi-debug-noexcept -lsockets  
wasm-ld: warning: function signature mismatch: cssyr_
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32) -> i32 in c_sblat2c.o
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32) -> void in c_sblas2.o

wasm-ld: warning: function signature mismatch: cs2chke_
>>> defined as (i32, i32) -> i32 in c_sblat2c.o
>>> defined as (i32) -> void in c_s2chke.o

wasm-ld: warning: function signature mismatch: csgbmv_
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32) -> i32 in c_sblat2c.o
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32, i32) -> void in c_sblas2.o

wasm-ld: warning: function signature mismatch: csspr2_
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32, i32) -> i32 in c_sblat2c.o
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32) -> void in c_sblas2.o

Here is the log with all the signature mismatch warnings:
signature-mismatch.log

[martin-frbg]

I saw them too, but I think they are "only" from missing or outdated declarations within the test/ctest/utest sources. At least they do not block building of the library - I still plan to get to them "later".

[lesteve]

OK makes sense, thanks a lot!