Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving
This work addresses safety and generalizability issues in autonomous driving agents, particularly in offline RL settings, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the challenge of maintaining safety in autonomous driving under diverse and unforeseen scenarios by introducing FUSION, a safety-aware causal representation method for offline reinforcement learning, which demonstrates a good balance between safety cost and utility reward compared to state-of-the-art baselines.
In the domain of autonomous driving, the offline Reinforcement Learning~(RL) approaches exhibit notable efficacy in addressing sequential decision-making problems from offline datasets. However, maintaining safety in diverse safety-critical scenarios remains a significant challenge due to long-tailed and unforeseen scenarios absent from offline datasets. In this paper, we introduce the saFety-aware strUctured Scenario representatION (FUSION), a pioneering representation learning method in offline RL to facilitate the learning of a generalizable end-to-end driving policy by leveraging structured scenario information. FUSION capitalizes on the causal relationships between the decomposed reward, cost, state, and action space, constructing a framework for structured sequential reasoning in dynamic traffic environments. We conduct extensive evaluations in two typical real-world settings of the distribution shift in autonomous vehicles, demonstrating the good balance between safety cost and utility reward compared to the current state-of-the-art safe RL and IL baselines. Empirical evidence in various driving scenarios attests that FUSION significantly enhances the safety and generalizability of autonomous driving agents, even in the face of challenging and unseen environments. Furthermore, our ablation studies reveal noticeable improvements in the integration of causal representation into the offline safe RL algorithm. Our code implementation is available at: https://sites.google.com/view/safe-fusion/.