An Attention Mechanism for Robust Multimodal Integration in a Global Workspace Architecture
This addresses the challenge of robust multimodal integration for AI systems, though it appears incremental as it builds on existing global workspace implementations.
The paper tackles the problem of improving noise robustness and generalization in multimodal integration by proposing a top-down attention mechanism for a global workspace architecture, achieving competitive state-of-the-art results on the MM-IMDb 1.0 benchmark.
Global Workspace Theory (GWT), inspired by cognitive neuroscience, posits that flexible cognition could arise via the attentional selection of a relevant subset of modalities within a multimodal integration system. This cognitive framework can inspire novel computational architectures for multimodal integration. Indeed, recent implementations of GWT have explored its multimodal representation capabilities, but the related attention mechanisms remain understudied. Here, we propose and evaluate a top-down attention mechanism to select modalities inside a global workspace. First, we demonstrate that our attention mechanism improves noise robustness of a global workspace system on two multimodal datasets of increasing complexity: Simple Shapes and MM-IMDb 1.0. Second, we highlight various cross-task and cross-modality generalization capabilities that are not shared by multimodal attention models from the literature. Comparing against existing baselines on the MM-IMDb 1.0 benchmark, we find our attention mechanism makes the global workspace competitive with the state of the art.