On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression
This work addresses loss function selection for regression tasks in deep learning, offering incremental theoretical insights and practical validation.
The paper tackles the problem of using mean absolute error (MAE) as a loss function for deep neural network-based vector-to-vector regression, showing that MAE provides performance bounds and is more appropriate than mean squared error (MSE), with speech enhancement experiments validating its advantages.
In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression. First, we show that a generalized upper-bound for DNN-based vector- to-vector regression can be ensured by leveraging the known Lipschitz continuity property of MAE. Next, we derive a new generalized upper bound in the presence of additive noise. Finally, in contrast to conventional MSE commonly adopted to approximate Gaussian errors for regression, we show that MAE can be interpreted as an error modeled by Laplacian distribution. Speech enhancement experiments are conducted to corroborate our proposed theorems and validate the performance advantages of MAE over MSE for DNN based regression.