Error compiling numpy to webassembly via emscrypten

[leopsidom]

I'm trying to compile numpy to webassembly using emscrypten provided by pyodide build tool, but I'm getting the following issue:

numpy/core/src/common/simd/avx512/avx512.h:54:18: error: unknown type name '__m512'; did you mean '__m128'?
typedef struct { __m512 val[3]; } npyv_f32x3;

I think it has something to do with numpy added avx512 support since 1.19+: 5562a8c. I'm wondering what compiler option we need to add after this addition.

Btw pyodide has already had numpy 1.17.5 built in, and up to 1.18.5, it still builds fine. But since 1.19+, we had some issues, though different from different versions.

Error message:

In file included from build/src.linux-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.avx512_skx.c:21:
In file included from numpy/core/src/_simd/_simd.dispatch.c.src:3:
In file included from numpy/core/src/_simd/_simd_inc.h.src:5:
In file included from numpy/core/src/common/simd/simd.h:31:
numpy/core/src/common/simd/avx512/avx512.h:11:9: error: unknown type name '__m512i'
typedef __m512i npyv_u8;
^
numpy/core/src/common/simd/avx512/avx512.h:12:9: error: unknown type name '__m512i'
typedef __m512i npyv_s8;
^
numpy/core/src/common/simd/avx512/avx512.h:13:9: error: unknown type name '__m512i'
typedef __m512i npyv_u16;
^
numpy/core/src/common/simd/avx512/avx512.h:14:9: error: unknown type name '__m512i'
typedef __m512i npyv_s16;
^
numpy/core/src/common/simd/avx512/avx512.h:15:9: error: unknown type name '__m512i'
typedef __m512i npyv_u32;
^
numpy/core/src/common/simd/avx512/avx512.h:16:9: error: unknown type name '__m512i'
typedef __m512i npyv_s32;
^
numpy/core/src/common/simd/avx512/avx512.h:17:9: error: unknown type name '__m512i'
typedef __m512i npyv_u64;
^
numpy/core/src/common/simd/avx512/avx512.h:18:9: error: unknown type name '__m512i'
typedef __m512i npyv_s64;
^
numpy/core/src/common/simd/avx512/avx512.h:19:9: error: unknown type name '__m512'; did you mean '__m128'?
typedef __m512 npyv_f32;
^~~~~~
__m128
/src/emsdk/emsdk/upstream/emscripten/cache/sysroot/include/compat/xmmintrin.h:23:17: note: '__m128' declared here
typedef __f32x4 __m128;
^
In file included from build/src.linux-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.avx512_skx.c:21:
In file included from numpy/core/src/_simd/_simd.dispatch.c.src:3:
In file included from numpy/core/src/_simd/_simd_inc.h.src:5:
In file included from numpy/core/src/common/simd/simd.h:31:
numpy/core/src/common/simd/avx512/avx512.h:20:9: error: unknown type name '__m512d'
typedef __m512d npyv_f64;
^
numpy/core/src/common/simd/avx512/avx512.h:23:9: error: unknown type name '__mmask64'
typedef __mmask64 npyv_b8;
^
numpy/core/src/common/simd/avx512/avx512.h:24:9: error: unknown type name '__mmask32'
typedef __mmask32 npyv_b16;
^
numpy/core/src/common/simd/avx512/avx512.h:29:9: error: unknown type name '__mmask16'
typedef __mmask16 npyv_b32;
^
numpy/core/src/common/simd/avx512/avx512.h:30:9: error: unknown type name '__mmask8'
typedef __mmask8 npyv_b64;
^
numpy/core/src/common/simd/avx512/avx512.h:32:18: error: unknown type name '__m512i'
typedef struct { __m512i val[2]; } npyv_m512ix2;
^
numpy/core/src/common/simd/avx512/avx512.h:42:18: error: unknown type name '__m512i'
typedef struct { __m512i val[3]; } npyv_m512ix3;
^
numpy/core/src/common/simd/avx512/avx512.h:52:18: error: unknown type name '__m512'; did you mean '__m128'?
typedef struct { __m512 val[2]; } npyv_f32x2;
^~~~~~
__m128
/src/emsdk/emsdk/upstream/emscripten/cache/sysroot/include/compat/xmmintrin.h:23:17: note: '__m128' declared here
typedef __f32x4 __m128;
^
In file included from build/src.linux-x86_64-3.9/numpy/core/src/_simd/_simd.dispatch.avx512_skx.c:21:
In file included from numpy/core/src/_simd/_simd.dispatch.c.src:3:
In file included from numpy/core/src/_simd/_simd_inc.h.src:5:
In file included from numpy/core/src/common/simd/simd.h:31:
numpy/core/src/common/simd/avx512/avx512.h:53:18: error: unknown type name '__m512d'
typedef struct { __m512d val[2]; } npyv_f64x2;
^
numpy/core/src/common/simd/avx512/avx512.h:54:18: error: unknown type name '__m512'; did you mean '__m128'?
typedef struct { __m512 val[3]; } npyv_f32x3;
^~~~~~
__m128
/src/emsdk/emsdk/upstream/emscripten/cache/sysroot/include/compat/xmmintrin.h:23:17: note: '__m128' declared here
typedef __f32x4 __m128;
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

NumPy/Python version information:

numpy 1.20.0
python 3.9.5

[leopsidom]

Open issue at pyodide: pyodide/pyodide#1827

[seberg]

Have you explored the dispatching related compile options already?: https://numpy.org/devdocs/reference/simd/simd-optimizations.html#build-options-for-compilation I don't know pyodine/webassembly, but I guess you may have to restrict the dispatching or even only compile a baseline.

[leopsidom]

@seberg Thanks for sharing the reference. I probably should play with some of those options. Let me have a try :)

[leopsidom]

It seems emscripten only supports up to 128 bit AVX, and do not understand instrinsics beyond that. @seberg Do you know how we can pass the compilation arguments to emcc ? I tried emcc .... --cpu-baseline=avx and emcc ... --cpu-baseline="avx". But I'm getting the following error:

clang-13: error: unsupported option '--cpu-baseline="avx"'
clang-13: error: unsupported option '--cpu-dispatch="avx"'

[leopsidom]

Okay I think I figured out how to pass the compiler option to numpy via the build.py and the issue has resolved for me. Thanks @seberg for the pointer.