Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning
This work addresses the challenge of sample efficiency and generalization in multi-goal robotics by adapting decision transformers to offline RL, offering a practical solution for real-world applications.
The paper introduces a Goal-Conditioned Decision Transformer for offline multi-goal robotics, which incorporates goal states into sequence modeling to solve varying tasks from pre-collected data. It outperforms state-of-the-art online baselines on the Franka Emika Panda platform, showing robustness in sparse-reward settings.
Reinforcement learning (RL) in robotics faces significant hurdles regarding sample efficiency and generalization across varying goals. While Offline RL mitigates the need for costly online interactions, its integration with goal-conditioned policies and transformer-based architectures remains underexplored. We introduce a Goal-Conditioned Decision Transformer adapted for offline multi-goal robotics. By explicitly incorporating goal states into the sequence modeling framework, our approach efficiently solves varying tasks using only pre-collected data. We validate this method on a newly released offline dataset for the Franka Emika Panda platform. Experimental results demonstrate that our approach outperforms state-of-the-art online baselines in complex tasks and maintains robustness in sparse-reward settings, even with limited expert demonstrations.