Robust Dynamic Multi-Modal Data Fusion: A Model Uncertainty Perspective
This addresses robust data fusion for applications like robotics or autonomous systems where modality failures can occur, but it appears incremental as it builds on existing filtering techniques.
The paper tackles the problem of multi-modal data fusion under unexpected modality failures in nonlinear non-Gaussian dynamic processes by proposing a framework that uses modality 'usefulness' and dynamic model averaging with a particle filter, resulting in experimental outcomes that show it outperforms state-of-the-art methods.
This paper is concerned with multi-modal data fusion (MMDF) under unexpected modality failures in nonlinear non-Gaussian dynamic processes. An efficient framework to tackle this problem is proposed. In particular, a notion termed modality "\emph{usefulness}", which takes a value of 1 or 0, is used for indicating whether the observation of this modality is useful or not. For $n$ modalities involved, $2^n$ combinations of their "\emph{usefulness}" values exist. Each combination defines one hypothetical model of the true data generative process. Then the problem of concern is formalized as a task of nonlinear non-Gaussian state filtering under model uncertainty, which is addressed by a dynamic model averaging (DMA) based particle filter (PF) algorithm. This DMA algorithm employs $2^n$ models, while all models share the same state-transition function and a unique set of particle values. That makes its computational complexity only slightly larger than a single model based PF algorithm, especially for scenarios in which $n$ is small. Experimental results show that the proposed solution outperforms remarkably state-of-the-art methods. Code and data are available at https://github.com/robinlau1981/fusion.