Fixes for Binary & Ternary NNs with QKeras
Created by: thesps
Our conversion flow wasn't really working for QKeras Binary & Ternary NNs before (it did at some point, not sure when it broke...). This PR restores that flow. It also brings support for the threshold
parameter to ternary
quantizer for QActivation
, which we didn't have before.
Here's some code testing several example models, combining binary/ternary weight quantizers, binary/ternary activations (specified with functions, classes, and strings), alpha (weight scale factor), and BatchNormalization.
The models are created with random weights, then converted, and the output is compared vs QKeras evaluation.
On master branch the output is like:
0
...
Discrepancies 0 / 10000
1
...
Couldn't convert, compile, and predict model
2
...
Couldn't convert, compile, and predict model
3
...
Couldn't convert, compile, and predict model
4
...
Couldn't convert, compile, and predict model
5
...
Couldn't convert, compile, and predict model
6
...
Couldn't convert, compile, and predict model
With the PR the output is like:
0
...
Discrepancies 0 / 10000
1
...
Discrepancies 4 / 10000
2
...
Discrepancies 1 / 10000
3
...
Discrepancies 40 / 10000
4
...
Discrepancies 7 / 10000
5
...
Discrepancies 2 / 10000
6
...
Discrepancies 1 / 10000