Score-oriented loss (SOL) functions
This work addresses the challenge of improving forecasting performance for machine learning practitioners, but it is incremental as it builds on existing loss function engineering.
The paper tackles the problem of optimizing skill scores in binary classification by introducing a new class of loss functions defined on probabilistic confusion matrices, showing that the probability distribution function of confusion matrices significantly impacts score maximization outcomes.
Loss functions engineering and the assessment of forecasting performances are two crucial and intertwined aspects of supervised machine learning. This paper focuses on binary classification to introduce a class of loss functions that are defined on probabilistic confusion matrices and that allow an automatic and a priori maximization of the skill scores. The performances of these loss functions are validated during the training phase of two experimental forecasting problems, thus showing that the probability distribution function associated with the confusion matrices significantly impacts the outcome of the score maximization process.