Robust and Active Learning for Deep Neural Network Regression
This work addresses the challenge of enhancing regression accuracy in deep learning applications, but it appears incremental as it builds on existing active learning and fine-tuning techniques.
The paper tackles the problem of improving deep neural network regression by identifying local error maximizers using gradient-based methods and then fine-tuning or retraining the model through active learning, assuming access to a slower oracle for supervision.
We describe a gradient-based method to discover local error maximizers of a deep neural network (DNN) used for regression, assuming the availability of an "oracle" capable of providing real-valued supervision (a regression target) for samples. For example, the oracle could be a numerical solver which, operationally, is much slower than the DNN. Given a discovered set of local error maximizers, the DNN is either fine-tuned or retrained in the manner of active learning.