Do you know how NVDLA implement the non-linear activation functions

[bandjshen]

From the RTL Source code ,it use 9 bits as address for LUT , and it can 4*16bits data in parallel ;
but it divide it's table into LO and LE , what this means ? how to decide to use LO or LE ?
why it both get the LE[addr] and LE[addr+1] ?

there are also scale/shift/offset/bias after LUT , what this means ?

[JunningWu]

I will check those codes these days.

[JunningWu]

JunningWu/AIChip#11

[JunningWu]

The Single Data Point Processor (SDP) allows for the application of both linear and non-linear functions onto individual data points. This is commonly used immediately after convolution in CNN systems. The SDP provides native support for linear functions (e.g., simple bias and scaling) and uses lookup tables (LUTs) to implement non-linear functions. This combination supports most common activation functions as well as other element-wise operations including: ReLU, PReLU, precision scaling, batch normalization, bias addition, or other complex non-linear functions, such as a sigmoid or a hyperbolic tangent.

I guess U shall understand SDP's functions first, then U will know the RTL code very well.

[bandjshen]

i know which functions inside SDP , but for detail solution , how to design LUT to implement ELU/Sigmoid/Tanh is not know ;
I'm a hw engineer , the code to calculate the table may exist in driver code when this layer try to support Sigmoid for example ;
another question is , SDP is 16 pipeline in parrellel , but for EW(the non-linear activation) , it seems only 4 pipeline ; it will hurt the performance .

[bandjshen]

Hi, Junning
Do you know in BS/BN instance , there are register config port cfg_alu_algo_rsc_z[1:0]
what this bits means ?
and how about cfg_alu_shift_value_rsc_z[5:0] ?

Thanks

[weikun750]

The index is consist of 9 bit INT and 35 bit FRAC, so if the index is inbetween of 2 entrys, the output will be Y0FRAC + Y1(1<<35 - FRAC), that's why, the HW always implement the addr and addr+1 logics.