What GELU algorithm is used for LIBXSMM_MELTW_TYPE_UNARY_GELU?

[jgong5]

Is it tanh approximation?

[alheinecke]

No it's a direct minmax approximation.
https://github.com/libxsmm/libxsmm/blob/main/src/generator_common_x86.c#L204
https://github.com/libxsmm/libxsmm/blob/main/src/generator_common_x86.c#L714

[jgong5]

No it's a direct minmax approximation. https://github.com/libxsmm/libxsmm/blob/main/src/generator_common_x86.c#L204 https://github.com/libxsmm/libxsmm/blob/main/src/generator_common_x86.c#L714

Thanks. Will you support tanh approximation too?

[alheinecke]

we have no plans for this as workloads converge with the current implementation. However, I can help you with guiding/submitting a PR following the same scheme (setting up const, computing x^3 and then using the minimax for tanh).