Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression
This provides theoretical guarantees for error bounds in DNN regression, which is incremental for researchers and practitioners in machine learning.
The paper tackles the problem of bounding mean absolute error in deep neural network vector-to-vector regression by deriving theoretical upper bounds based on approximation, estimation, and optimization errors, and validates these bounds through experiments in image denoising and speech enhancement, showing they hold with and without over-parametrization.
In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error. Leveraging upon error decomposition techniques in statistical learning theory and non-convex optimization theory, we derive upper bounds for each of the three aforementioned errors and impose necessary constraints on DNN models. Moreover, we assess our theoretical results through a set of image de-noising and speech enhancement experiments. Our proposed upper bounds of MAE for DNN based vector-to-vector regression are corroborated by the experimental results and the upper bounds are valid with and without the "over-parametrization" technique.