System-Level Uncertainty Quantification with Multiple Machine Learning Models: A Theoretical Framework
This foundational work addresses uncertainty quantification for decision-making and design in systems using multiple ML models, but it is incremental as it builds on existing uncertainty concepts without introducing a new paradigm.
The study tackled the problem of quantifying uncertainty in predictions from multiple machine learning models by developing a theoretical framework that generates the joint distribution of predictions, accounting for dependent model uncertainties and random inputs, and decouples these uncertainties into independent variables.
ML models have errors when used for predictions. The errors are unknown but can be quantified by model uncertainty. When multiple ML models are trained using the same training points, their model uncertainties may be statistically dependent. In reality, model inputs are also random with input uncertainty. The effects of these types of uncertainty must be considered in decision-making and design. This study develops a theoretical framework that generates the joint distribution of multiple ML predictions given the joint distribution of model uncertainties and the joint distribution of model inputs. The strategy is to decouple the coupling between the two types of uncertainty and transform them as independent random variables. The framework lays a foundation for numerical algorithm development for various specific applications.