Skip to content

Softmax with invert LUT

Javier Duarte requested to merge jmgd/softmax-invert-lut into master

Modified the softmax activation function to use an invert LUT (default 10 bits from 0 to 64 - this might need to be tuned).

The latency for the softmax layer in the 50% pruned three-layer example goes down from 35 to 8, but it's still not as good as a simple sigmoid which is 2.

softmax:

+ Latency (clock cycles): 
    * Summary: 
    +-----+-----+-----+-----+----------+
    |  Latency  |  Interval | Pipeline |
    | min | max | min | max |   Type   |
    +-----+-----+-----+-----+----------+
    |   35|   35|    1|    1| function |
    +-----+-----+-----+-----+----------+

softmax with invert LUT:

+ Latency (clock cycles): 
    * Summary: 
    +-----+-----+-----+-----+----------+
    |  Latency  |  Interval | Pipeline |
    | min | max | min | max |   Type   |
    +-----+-----+-----+-----+----------+
    |    8|    8|    1|    1| function |
    +-----+-----+-----+-----+----------+

sigmoid:

+ Latency (clock cycles): 
    * Summary: 
    +-----+-----+-----+-----+----------+
    |  Latency  |  Interval | Pipeline |
    | min | max | min | max |   Type   |
    +-----+-----+-----+-----+----------+
    |    2|    2|    1|    1| function |
    +-----+-----+-----+-----+----------+

Merge request reports