I would also like to mention my own interest in having general support for int8 as a feature.

This is a great idea but we need to be careful to up/down convert on certain hardware when adding this feature. For example fp16 performance on compute_61(non-Tesla Pascal) cards is absolutely abysmal. We will also need to support this on older hardware. I wonder how cuda provides fall back in those cases.

AFAIK, there is no support for half precision floating point numbers in C and C++ standards. So fp16 is going to be a bit to support in a general purpose manner.

int8 can be supported easily (although it is a bit tedious).

Found the half library. It seems to be well documented. Probably need to run some tests on performance and compatibility with native types.

@umar456 Does boost have anything similar ?

Would int8 easier to implement than fp16, especially the int8 cuda8 pascal gpu backend ?

@WilliamTambellini yes int8 support is much easier to accomplish.

ok so we should better split this feature/issue in 2: int8 and fp16.
Would you mind ?

The issue with int8 (char) is that we use it for b8. That is a differentiation we will have to keep in mind.

@shehzan10 int8_t is a different datatype which can be used for i8

Ok thanks.
WARNING: I have renamed this feature to "int8" in order to limit the scope to int8 support. Anyone interested by fp16 should create another github issue/ticket.
Cheers
W.

I do confirm our interest in int8 mainly via cuda.
Could anyone summarize which part of ArrayFire would need to be modified in order to take advantage of INT8 hardware acceleration ?
Cheers

@WilliamTambellini This involves changing a lot of files across 3 backends. This is not fairly straight forward.

I'd just like to add my support for int8 support.

Seen here:
https://devblogs.nvidia.com/parallelforall/mixed-precision-programming-cuda-8/
"cuBLAS is a GPU library for dense linear algebra— an implementation of BLAS, the Basic Linear Algebra Subroutines. cuBLAS has support for mixed precision in several matrix-matrix multiplication routines. cublasHgemm is a FP16 dense matrix-matrix multiply routine that uses FP16 for compute as well as for input and output. cublasSgemmEx() computes in FP32, but the input data can be FP32, FP16, or INT8, and the output can be FP32 or FP16. cublasGemm() is a new routine in CUDA 8 that allows specification of the computation precision, including INT8 computation (which uses DP4A)."

@umar456 would it be possible to do a minimalist implementation in AF in order to call cublasGemm() when af array datatype is int8 or int16 ?
Tks

Attaching a minimalist poc (seen on the nvidia forum) of int8 matmul using cublasGemmEx :
int8cublas.cu.txt
TBC.

@umar456 this one is not needed for 3.7.0 on my side.

Could we please remove that one from the 3.7.0 scope ?

up ?