Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing
This addresses the issue of imbalanced label distributions in regression tasks for machine learning practitioners, offering improved performance and uncertainty quantification, though it appears incremental as it builds on existing variational and probabilistic methods.
The paper tackles the problem of poor accuracy and uncertainty estimation in regression models when label distributions are imbalanced, proposing a probabilistic deep learning model called variational imbalanced regression (VIR) that outperforms state-of-the-art methods in both accuracy and uncertainty estimation on real-world datasets.
Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representations (a data point's representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation. Code will soon be available at https://github.com/Wang-ML-Lab/variational-imbalanced-regression.