Output-weighted and relative entropy loss functions for deep learning precursors of extreme events
This addresses the problem of imbalanced datasets in scientific and engineering applications where standard machine learning methods fail to predict extreme events, though it is incremental as it builds on existing loss function approaches.
The paper tackles the challenge of predicting rare extreme events in dynamical systems by proposing new loss functions, the adjusted output weighted loss and relative entropy based loss, which significantly improve accuracy for these events in tested cases.
Many scientific and engineering problems require accurate models of dynamical systems with rare and extreme events. Such problems present a challenging task for data-driven modelling, with many naive machine learning methods failing to predict or accurately quantify such events. One cause for this difficulty is that systems with extreme events, by definition, yield imbalanced datasets and that standard loss functions easily ignore rare events. That is, metrics for goodness of fit used to train models are not designed to ensure accuracy on rare events. This work seeks to improve the performance of regression models for extreme events by considering loss functions designed to highlight outliers. We propose a novel loss function, the adjusted output weighted loss, and extend the applicability of relative entropy based loss functions to systems with low dimensional output. The proposed functions are tested using several cases of dynamical systems exhibiting extreme events and shown to significantly improve accuracy in predictions of extreme events.