New testing
Created by: thesps
This PR adds pytest
tests and Gitlab CI file to run those tests on the CERN Gitlab infrastructure. This is intended to run in addition to the current Jenkins set up for now.
The tests currently included in this new set extend our current testing to execute inference of models and require "good enough" performance at the model output vs some Python reference. There are some completely synthetic models running on random data with a requirement on the relative and absolute difference at the model output, and some real models (using updated example-models
) with a requirement on the model accuracy. One set of tests also checks attributes of the original Keras model vs the HLSModel to check that the conversion is correct directly.
Several of these tests currently fail! The intention is to fix the issues causing these tests to fail separately from this PR, but as a matter of priority. Some of them are actually already fixed in existing PRs.
Also worth noting: the tests currently implemented are not the final set, we should continue to add more.