Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition
This addresses uncertainty reduction for AI systems using multi-modal data, but it appears incremental as it builds on existing ideas like active learning and uncertainty quantification.
The paper tackles the challenge of disentangling uncertainty in multi-modal AI systems by introducing a data acquisition framework that samples across modalities and observations, hypothesizing that aleatoric uncertainty decreases with more modalities and epistemic uncertainty with more data, with proof-of-concept implementations on two datasets.
To generate accurate and reliable predictions, modern AI systems need to combine data from multiple modalities, such as text, images, audio, spreadsheets, and time series. Multi-modal data introduces new opportunities and challenges for disentangling uncertainty: it is commonly assumed in the machine learning community that epistemic uncertainty can be reduced by collecting more data, while aleatoric uncertainty is irreducible. However, this assumption is challenged in modern AI systems when information is obtained from different modalities. This paper introduces an innovative data acquisition framework where uncertainty disentanglement leads to actionable decisions, allowing sampling in two directions: sample size and data modality. The main hypothesis is that aleatoric uncertainty decreases as the number of modalities increases, while epistemic uncertainty decreases by collecting more observations. We provide proof-of-concept implementations on two multi-modal datasets to showcase our data acquisition framework, which combines ideas from active learning, active feature acquisition and uncertainty quantification.