SALSA-RL: Stability Analysis in the Latent Space of Actions for Reinforcement Learning
This addresses the need for interpretable and safe RL in control systems, though it is incremental as it builds on existing methods for stability analysis.
The paper tackles the problem of interpretability and safety in deep reinforcement learning for real-world control systems by proposing SALSA-RL, a framework that models actions in a latent space to enable local stability analysis, resulting in non-invasive assessment without performance loss across benchmarks.
Modern deep reinforcement learning (DRL) methods have made significant advances in handling continuous action spaces. However, real-world control systems--especially those requiring precise and reliable performance--often demand interpretability in the sense of a-priori assessments of agent behavior to identify safe or failure-prone interactions with environments. To address this limitation, we propose SALSA-RL (Stability Analysis in the Latent Space of Actions), a novel RL framework that models control actions as dynamic, time-dependent variables evolving within a latent space. By employing a pre-trained encoder-decoder and a state-dependent linear system, our approach enables interpretability through local stability analysis, where instantaneous growth in action-norms can be predicted before their execution. We demonstrate that SALSA-RL can be deployed in a non-invasive manner for assessing the local stability of actions from pretrained RL agents without compromising on performance across diverse benchmark environments. By enabling a more interpretable analysis of action generation, SALSA-RL provides a powerful tool for advancing the design, analysis, and theoretical understanding of RL systems.