MCAM: Multimodal Causal Analysis Model for Ego-Vehicle-Level Driving Video Understanding
This work addresses the need for accurate driving behavior recognition and reasoning in autonomous driving applications, representing an incremental improvement over existing methods.
The paper tackles the problem of shallow causal reasoning and spurious correlations in autonomous driving video understanding by proposing MCAM, a multimodal causal analysis model that constructs latent causal structures between visual and language modalities, achieving state-of-the-art performance on BDD-X and CoVLA datasets.
Accurate driving behavior recognition and reasoning are critical for autonomous driving video understanding. However, existing methods often tend to dig out the shallow causal, fail to address spurious correlations across modalities, and ignore the ego-vehicle level causality modeling. To overcome these limitations, we propose a novel Multimodal Causal Analysis Model (MCAM) that constructs latent causal structures between visual and language modalities. Firstly, we design a multi-level feature extractor to capture long-range dependencies. Secondly, we design a causal analysis module that dynamically models driving scenarios using a directed acyclic graph (DAG) of driving states. Thirdly, we utilize a vision-language transformer to align critical visual features with their corresponding linguistic expressions. Extensive experiments on the BDD-X, and CoVLA datasets demonstrate that MCAM achieves SOTA performance in visual-language causal relationship learning. Furthermore, the model exhibits superior capability in capturing causal characteristics within video sequences, showcasing its effectiveness for autonomous driving applications. The code is available at https://github.com/SixCorePeach/MCAM.