Grace / V2

[breuera]

NVIDIA Grace is not discovered properly:

LIBXSMM ERROR (libxsmm_generator_mateltwise_aarch64_update_micro_kernel_config_vectorlength): unknown architecture (error #90005)!
M blocking found is 2
LIBXSMM ERROR (libxsmm_generator_gemm_kernel): unknown architecture or unsupported precision (error #90004)!
LIBXSMM ERROR (libxsmm_generator_mateltwise_aarch64_update_micro_kernel_config_vectorlength): unknown architecture (error #90005)!
M blocking found is 3
LIBXSMM ERROR (libxsmm_generator_gemm_kernel): unknown architecture or unsupported precision (error #90004)!

I believe that 128-bit SVE is not operational right now. Is somebody working on that?

In either case: Discovering Neon as target should be a simple fix; setting LIBXSMM_TARGET=aarch64 works perfectly fine. Is somebody working on that already?

[alheinecke]

No, right now nobody is working on SVE128 or V2 support, simply as we don't have access to hardware. Any help on this welcome :-)

[FreddieWitherden]

This will likely require some work on the asparse_reg side too as we would want to consider both NEON and SVE128 kernels (as it is not clean that 128-bit SVE will outperform NEON whereas 256-bit SVE is quite clear cut).

[breuera]

Here's a full sweep for matrix-matrix mult. using ASIMD/Neon and FP32 for M,N,K in [1, ..., 64].
In the plots K is fixed to K=64 (raw data below).

Fixing N=6:

Fixing N=32:

Fixing N=64:

Overall the performance seems to be too low.
Achievable peak is at about 105 FP32 GFLOPS.
NVPL BLAS GEMM gets that (for way larger matrices, M,N,K >~1024).

We'll now have a look at how to write a fast assembly kernel for a single case.
This should help us optimizing the 128-bit JITters.
I'd also doubt that SVE128 probably helps much over Neon.

perf_grid_grace_sgemm_xsmm.csv

[FreddieWitherden]

Do we know the FMA issue rate and the latency for back to back accumulations? This will tell us how many accumulators we need to use and can compare that against what the routine currently uses.

[breuera]

I am only aware of the numbers in the software optimization guides: https://developer.arm.com/documentation/PJDOC-466751330-593177/latest/

[alheinecke]

I believe the ASIMD kernel is using in register broadcast, doesn't it? Perhaps the x86 style explicit broadcast and full SIMD FMA is faster?

[stefan0re]

Currently i have an assembly kernel for NEON on the Grace CPU that achieves 102 GFLOPS on small fixed-size matrix, closely approaching the peak performance of 105 GFLOPS.

[FreddieWitherden]

I believe the ASIMD kernel is using in register broadcast, doesn't it? Perhaps the x86 style explicit broadcast and full SIMD FMA is faster?

On A64FX I observed a heavy penalty for using the register broadcasting. However, the ARM-designed cores seem to handle this quite well.

[breuera]

Haven't seen bcasting issues on Neoverse either. But might be worth to double-check for V2.