The Earth Mover's Pinball Loss: Quantiles for Histogram-Valued Regression
This addresses a gap in deep learning for histogram data, benefiting scientific fields like astrophysics and sports analytics, though it is an incremental advancement in loss function design.
The paper tackles the problem of histogram-valued regression in deep learning by introducing a new loss function that generalizes the Earth Mover's distance to arbitrary quantiles, enabling predictions of distributions over histograms. The method achieves accuracy similar to standard EMD for median histograms and provides more informative predictions at minimal computational cost.
Although ubiquitous in the sciences, histogram data have not received much attention by the Deep Learning community. Whilst regression and classification tasks for scalar and vector data are routinely solved by neural networks, a principled approach for estimating histogram labels as a function of an input vector or image is lacking in the literature. We present a dedicated method for Deep Learning-based histogram regression, which incorporates cross-bin information and yields distributions over possible histograms, expressed by $τ$-quantiles of the cumulative histogram in each bin. The crux of our approach is a new loss function obtained by applying the pinball loss to the cumulative histogram, which for 1D histograms reduces to the Earth Mover's distance (EMD) in the special case of the median ($τ= 0.5$), and generalizes it to arbitrary quantiles. We validate our method with an illustrative toy example, a football-related task, and an astrophysical computer vision problem. We show that with our loss function, the accuracy of the predicted median histograms is very similar to the standard EMD case (and higher than for per-bin loss functions such as cross-entropy), while the predictions become much more informative at almost no additional computational cost.