On the inconsistency of separable losses for structured prediction
This reveals a fundamental inconsistency in widely used losses for structured prediction, potentially impacting researchers and practitioners in machine learning.
The paper proves that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, meaning minimizing them may not yield models that predict the most probable structure from the data distribution.
In this paper, we prove that separable negative log-likelihood losses for structured prediction are not necessarily Bayes consistent, or, in other words, minimizing these losses may not result in a model that predicts the most probable structure in the data distribution for a given input. This fact opens the question of whether these losses are well-adapted for structured prediction and, if so, why.