Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies
This work addresses adaptation challenges in dynamic real-world scenarios like robotics assembly lines, but it is incremental as it applies an existing method to new data.
The paper tackled the problem of applying Diffusion Policy to non-stationary, vision-based reinforcement learning tasks, such as robotics and navigation, and found that it consistently outperformed standard RL methods like PPO and DQN by achieving higher mean and maximum rewards with reduced variability.
This paper investigates the application of Diffusion Policy in non-stationary, vision-based RL settings, specifically targeting environments where task dynamics and objectives evolve over time. Our work is grounded in practical challenges encountered in dynamic real-world scenarios such as robotics assembly lines and autonomous navigation, where agents must adapt control strategies from high-dimensional visual inputs. We apply Diffusion Policy -- which leverages iterative stochastic denoising to refine latent action representations-to benchmark environments including Procgen and PointMaze. Our experiments demonstrate that, despite increased computational demands, Diffusion Policy consistently outperforms standard RL methods such as PPO and DQN, achieving higher mean and maximum rewards with reduced variability. These findings underscore the approach's capability to generate coherent, contextually relevant action sequences in continuously shifting conditions, while also highlighting areas for further improvement in handling extreme non-stationarity.