Skip to content

Per-layer precision and reuse factor

Javier Duarte requested to merge github/fork/vloncar/precision into master

Created by: vloncar

Currently, precision is defined on a per-project basis. The PR expands the granularity of precision specification to enable per-layer precision, but still retains the option of a more coarse-grained specification.

Instead of using a single DefaultPrecision for everything, precision can be defined on three levels, Model, LayerType and LayerName. Precision defined within Model will apply to all layers within the model, that don't have a precision specified. LayerType will apply to layers of specific kind (Dense, Conv1D...), while LayerName will apply to the specific layer, given its name. At the layer level, it is possible to define precision for all arrays involved (e.g., for Dense layer, precision can be different for the resulting tensor, for weights, bias and intermediate accum arrays).

To ease the definition of precision, both the full and shorter synthax are accepted. For example:

# Applies to the whole model, nothing else can be defined (mimics current behavior)
Precision: ap_fixed<16,6>

# Everything not explicitly specified will use the default
Precision:
  Model: ap_fixed<16,6> 
  ...

# All weights not explicitly specified will use the default weight precision,
# everything else not explicitly specified will use the default precision
Precision:
  Model:
    default: ap_fixed<16,6>
    weight: ap_fixed<14,6>
    ...
  ...

A more concrete example for three_layer model:

Precision:
  Model: ap_fixed<18,8>
  LayerType:
    Dense:
      default: ap_fixed<16,6>
      weight: ap_fixed<14,6>
    Activation: ap_fixed<8,4>
  LayerName:
    fc1_relu:
      weight: ap_fixed<18,6>
      bias: ap_fixed<16,8>
      result: ap_fixed<18,8>
    fc3_relu:
      default: ap_fixed<14,6>
      bias: ap_fixed<10,4>

(Please ignore the fact that this is nonsense for a moment, as no real model would use precision so different for each layer)

In this example, first layer, named fc1_relu will use the specified precision for its weights, bias and result, while the intermediate results (accum) will use default model precision (ap_fixed<18,8>). Not being explicitly specified, the second Dense layer (fc2_relu) will use the precision defined for its layer type. The third layer will use ap_fixed<14,6> for everything except for the bias, which will use ap_fixed<10,4>. Additionally, all activation layers will use ap_fixed<8,4>

Merge request reports