Multi-Modal Mutual Information (MuMMI) Training for Robust Self-Supervised Deep Reinforcement Learning
This work addresses the issue of poor performance and sensor over-reliance in multi-modal deep reinforcement learning, offering a solution for more robust AI systems in tasks like robotics.
The paper tackles the problem of learning robust deep world models from multiple unreliable sensors by introducing a multi-modal deep latent state-space model trained with a mutual information lower-bound, which significantly outperforms state-of-the-art deep reinforcement learning methods, especially when observations are missing.
This work focuses on learning useful and robust deep world models using multiple, possibly unreliable, sensors. We find that current methods do not sufficiently encourage a shared representation between modalities; this can cause poor performance on downstream tasks and over-reliance on specific sensors. As a solution, we contribute a new multi-modal deep latent state-space model, trained using a mutual information lower-bound. The key innovation is a specially-designed density ratio estimator that encourages consistency between the latent codes of each modality. We tasked our method to learn policies (in a self-supervised manner) on multi-modal Natural MuJoCo benchmarks and a challenging Table Wiping task. Experiments show our method significantly outperforms state-of-the-art deep reinforcement learning methods, particularly in the presence of missing observations.