Repository to show case simple and minimal testing of ML systems part of demo for KaggleX
Below follows a quick overview of some of the tests in this repo, for more details see this blog post
These type of tests are used as different sanity checks to identify bugs early on in the development process of a ML system.
Some example tests, with examples here (✅):
- Check shape of model output ✅
- Check shape of model input ✅
- Check output ranges ✅
- Make assertions on your dataset
- Check for data leakage between your datasets ✅
These type of tests do normally fall into two different groups: invariance tests & directional expectation tests
Some example tests, with examples here (✅):
- Assert that model output consistent to small changes in a feature of interest :check:
Tests to check for potential drift in input data, some tests to consider:
- T-test
- Kolmogorov-Smirnov test ✅
- Kullback–Leibler divergence
- Install vs-code
- Install the dev-containers extension
- Use the
DockerFilein this repository andRe-openas the container:
To run all tests in this repository run the below in the dev-container:
python -m pytest --disable-pytest-warnings src/ --no-header -v