PROPEL: Probabilistic Parametric Regression Loss for Convolutional Neural Networks
This addresses the challenge of probabilistic regression in computer vision for applications like orientation estimation, though it is incremental as it builds on existing CNN architectures.
The paper tackles the problem of applying probabilistic methods to regression tasks in CNNs, which typically handle unconstrained continuous variables, by proposing PROPEL, a loss function that enables learning parameters of probability distributions. The result shows that PROPEL improves accuracy and reduces model parameters by 10x compared to state-of-the-art methods in hand and head orientation estimation.
In recent years, Convolutional Neural Networks (CNNs) have enabled significant advancements to the state-of-the-art in computer vision. For classification tasks, CNNs have widely employed probabilistic output and have shown the significance of providing additional confidence for predictions. However, such probabilistic methodologies are not widely applicable for addressing regression problems using CNNs, as regression involves learning unconstrained continuous and, in many cases, multi-variate target variables. We propose a PRObabilistic Parametric rEgression Loss (PROPEL) that facilitates CNNs to learn parameters of probability distributions for addressing probabilistic regression problems. PROPEL is fully differentiable and, hence, can be easily incorporated for end-to-end training of existing CNN regression architectures using existing optimization algorithms. The proposed method is flexible as it enables learning complex unconstrained probabilities while being generalizable to higher dimensional multi-variate regression problems. We utilize a PROPEL-based CNN to address the problem of learning hand and head orientation from uncalibrated color images. Our experimental validation and comparison with existing CNN regression loss functions show that PROPEL improves the accuracy of a CNN by enabling probabilistic regression, while significantly reducing required model parameters by $10 \times$, resulting in improved generalization as compared to the existing state-of-the-art.