Skip to content

Added io_stream activation function implementations

Javier Duarte requested to merge github/fork/veyron8800/cnn_stream_dev into master

Created by: veyron8800

Hello,

This PR adds support for additional activation functions with the new io_stream option from PR #220 . These are activations that were previously supported with io_series and io_parallel. They are as follows:

  1. TanH
  2. Hard Sigmoid
  3. Leaky ReLU
  4. Thresholded ReLU
  5. Softplus
  6. Softsign
  7. ELU
  8. SeLU
  9. PReLU

Their implementations are validated through a test script which can be found here.

Also of note, I switched around the result writing order in the C++ test bench to correct a bug. If data file paths are provided in the yaml configuration, then writing results to disk is done before printing to console. However, keeping data in the output stream is only done when printing to console. This caused no results to be printed at the checkpoint, because the stream was emptied out by writing to the log on that iteration. Reversing the order of these tasks allows data to be printed to console, while also being written to file. This also ensures that there is no leftover data in the stream that has not been read by the test bench.

Thanks

Merge request reports