Training of Neural Networks with Uncertain Data: A Mixture of Experts Approach
This addresses uncertainty management in data-driven domains like biomedical signal processing and autonomous driving, but appears incremental as it builds on mixture of experts approaches.
The paper tackles the problem of aleatoric uncertainty in neural networks by introducing the Uncertainty-aware Mixture of Experts (uMoE), which embeds uncertainty into training and partitions input space, resulting in superior performance over baseline methods with demonstrated adaptability to varying uncertainty levels.
This paper introduces the "Uncertainty-aware Mixture of Experts" (uMoE), a novel solution aimed at addressing aleatoric uncertainty within Neural Network (NN) based predictive models. While existing methodologies primarily concentrate on managing uncertainty during inference, uMoE uniquely embeds uncertainty into the training phase. Employing a "Divide and Conquer" strategy, uMoE strategically partitions the uncertain input space into more manageable subspaces. It comprises Expert components, individually trained on their respective subspace uncertainties. Overarching the Experts, a Gating Unit, leveraging additional information regarding the distribution of uncertain in-puts across these subspaces, dynamically adjusts the weighting to minimize deviations from ground truth. Our findings demonstrate the superior performance of uMoE over baseline methods in effectively managing data uncertainty. Furthermore, through a comprehensive robustness analysis, we showcase its adaptability to varying uncertainty levels and propose optimal threshold parameters. This innovative approach boasts broad applicability across diverse da-ta-driven domains, including but not limited to biomedical signal processing, autonomous driving, and production quality control.