Improving Deep Regression with Ordinal Entropy
This work addresses a performance gap in regression tasks for computer vision, offering a method to enhance feature learning, though it appears incremental as it builds on known classification-regression comparisons.
The paper tackles the problem of improving deep regression by analyzing why classification often outperforms regression, and proposes an ordinal entropy loss to increase feature entropy while preserving ordinal relationships, showing benefits in experiments on synthetic and real-world tasks.
In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance. We investigate this curious phenomenon and provide a derivation to show that classification, with the cross-entropy loss, outperforms regression with a mean squared error loss in its ability to learn high-entropy feature representations. Based on the analysis, we propose an ordinal entropy loss to encourage higher-entropy feature spaces while maintaining ordinal relationships to improve the performance of regression tasks. Experiments on synthetic and real-world regression tasks demonstrate the importance and benefits of increasing entropy for regression.