AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks
This addresses the issue of suboptimal performance and overfitting in regression tasks for applications like recommendation systems and time series forecasting, though it appears incremental as it builds on existing pairwise and probabilistic methods.
The paper tackles the problem of deep regression models neglecting relative ordering and aleatoric uncertainty by proposing AdaPRL, an adaptive pairwise learning framework that integrates uncertainty estimation, achieving state-of-the-art performance on multiple real-world datasets without extra inference cost.
Current deep regression models usually learn in a point-wise way that treats each sample as an independent input, neglecting the relative ordering among different data. Consequently, the regression model could neglect the data's interrelationships, potentially resulting in suboptimal performance. Moreover, the existence of aleatoric uncertainty in the training data may drive the model to capture non-generalizable patterns, contributing to increased overfitting. To address these issues, we propose a novel adaptive pairwise learning framework for regression tasks (AdaPRL) which leverages the relative differences between data points and integrates with deep probabilistic models to quantify the uncertainty associated with the predictions. Additionally, we adapt AdaPRL for applications in multi-task learning and multivariate time series forecasting. Extensive experiments with several real-world regression datasets including recommendation systems, age prediction, time series forecasting, natural language understanding, finance, and industry datasets show that AdaPRL is compatible with different backbone networks in various tasks and achieves state-of-the-art performance on the vast majority of tasks without extra inference cost, highlighting its notable potential including enhancing prediction accuracy and ranking ability, increasing generalization capability, improving robustness to noisy data, improving resilience to reduced data, and enhancing interpretability. Experiments also show that AdaPRL can be seamlessly incorporated into recently proposed regression frameworks to gain performance improvement.