Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving
This work addresses the challenge of improving safety and decision-making in autonomous vehicles, though it appears incremental as it builds on existing machine learning methods.
The paper tackles the problem of enabling autonomous vehicles to learn self-awareness models from human driving data, using synchronized multi-sensor inputs to discover contextual multi-modal concepts, with results demonstrating potential for anomaly detection and autonomous decision-making.
This paper presents a novel approach for learning self-awareness models for autonomous vehicles. The proposed technique is based on the availability of synchronized multi-sensor dynamic data related to different maneuvering tasks performed by a human operator. It is shown that different machine learning approaches can be used to first learn single modality models using coupled Dynamic Bayesian Networks; such models are then correlated at event level to discover contextual multi-modal concepts. In the presented case, visual perception and localization are used as modalities. Cross-correlations among modalities in time is discovered from data and are described as probabilistic links connecting shared and private multi-modal DBNs at the event (discrete) level. Results are presented on experiments performed on an autonomous vehicle, highlighting potentiality of the proposed approach to allow anomaly detection and autonomous decision making based on learned self-awareness models.