ACCon: Angle-Compensated Contrastive Regularizer for Deep Regression
This addresses a fundamental problem in deep regression for improving model performance, especially in imbalanced or limited-data scenarios, but is incremental as it builds on existing contrastive learning methods.
The paper tackles the challenge of capturing relationships among continuous labels in deep regression by proposing an angle-compensated contrastive regularizer, which achieves competitive performance and excels in data efficiency and effectiveness on imbalanced datasets.
In deep regression, capturing the relationship among continuous labels in feature space is a fundamental challenge that has attracted increasing interest. Addressing this issue can prevent models from converging to suboptimal solutions across various regression tasks, leading to improved performance, especially for imbalanced regression and under limited sample sizes. However, existing approaches often rely on order-aware representation learning or distance-based weighting. In this paper, we hypothesize a linear negative correlation between label distances and representation similarities in regression tasks. To implement this, we propose an angle-compensated contrastive regularizer for deep regression, which adjusts the cosine distance between anchor and negative samples within the contrastive learning framework. Our method offers a plug-and-play compatible solution that extends most existing contrastive learning methods for regression tasks. Extensive experiments and theoretical analysis demonstrate that our proposed angle-compensated contrastive regularizer not only achieves competitive regression performance but also excels in data efficiency and effectiveness on imbalanced datasets.