Variation-Incentive Loss Re-weighting for Regression Analysis on Biased Data
This addresses data skewness in regression analysis, which is a domain-specific problem for machine learning practitioners, and is incremental as it builds on existing loss re-weighting techniques.
The paper tackles the problem of biased training data in regression tasks by proposing a Variation-Incentive Loss re-weighting method (VILoss) that uses uniqueness and abnormality metrics to optimize training, resulting in up to an 11.9% reduction in error.
Both classification and regression tasks are susceptible to the biased distribution of training data. However, existing approaches are focused on the class-imbalanced learning and cannot be applied to the problems of numerical regression where the learning targets are continuous values rather than discrete labels. In this paper, we aim to improve the accuracy of the regression analysis by addressing the data skewness/bias during model training. We first introduce two metrics, uniqueness and abnormality, to reflect the localized data distribution from the perspectives of their feature (i.e., input) space and target (i.e., output) space. Combining these two metrics we propose a Variation-Incentive Loss re-weighting method (VILoss) to optimize the gradient descent-based model training for regression analysis. We have conducted comprehensive experiments on both synthetic and real-world data sets. The results show significant improvement in the model quality (reduction in error by up to 11.9%) when using VILoss as the loss criterion in training.