Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies
For researchers in robot learning and interpretability, this work provides a first-of-its-kind method to ground SAE-based analysis in closed-loop behavioral events, addressing the unique challenge of interpreting VLA policies where outputs are actions rather than tokens.
The paper introduces an event-grounded interpretability pipeline for Vision-Language-Action (VLA) policies, linking sparse autoencoder (SAE) features to behavioral events rather than text contexts. Across simulations and real-robot studies, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to π0.5, demonstrating a practical starting point for behavior-anchored VLA interpretability.
Vision-Language-Action (VLA) policies translate language and visual inputs into robot actions, where their hidden representations directly shape closed-loop behavior. However, mechanistic interpretability tools from language and vision-language models do not transfer cleanly to VLAs: outputs are robot actions rather than human-readable tokens, and interventions can only be tested via expensive closed-loop rollouts. We propose an event-grounded interpretability pipeline that anchors SAE feature analysis to behavioral events rather than text contexts. End-effector keyframes are clustered within each task using visual, state, and temporal cues, linking SAE features to behaviorally salient events and, via optional VLM annotations, to semantic context. To our knowledge, our pipeline is among the first to ground SAE-based VLA analysis in closed-loop behavioral events. Across two simulation architectures and a real-robot study, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to the continuous action chunks of $π_{0.5}$. SAE is a sparse but imperfect intervention basis: usability varies with architecture and intervention site, and aggressive intervention reveals safety and interpretability limits. Overall, event-grounded SAE analysis emerges as a practical starting point for behavior-anchored VLA interpretability, motivating future work on SAE features beyond action-aligned coordinates, finer-grained closed-loop evaluation, and safe interventions for high-stakes VLA deployments. Code is available at \url{https://github.com/xc-j/Event-SAE}.