Towards Testing of Deep Learning Systems with Training Set Reduction
This addresses testing efficiency for developers of deep learning systems, but it is incremental as it builds on existing testing practices.
The paper tackles the problem of resource-intensive testing of deep learning training routines by evaluating training set reduction methods to mimic full-data training characteristics, finding it useful in resource-constrained environments.
Testing the implementation of deep learning systems and their training routines is crucial to maintain a reliable code base. Modern software development employs processes, such as Continuous Integration, in which changes to the software are frequently integrated and tested. However, testing the training routines requires running them and fully training a deep learning model can be resource-intensive, when using the full data set. Using only a subset of the training data can improve test run time, but can also reduce its effectiveness. We evaluate different ways for training set reduction and their ability to mimic the characteristics of model training with the original full data set. Our results underline the usefulness of training set reduction, especially in resource-constrained environments.