Residual Reinforcement Learning for Robot Teleoperation under Stochastic Delays
For robot teleoperation systems, this hybrid approach addresses the practical problem of stochastic communication delays, enabling stable and accurate control.
Stochastic delays in teleoperation cause signal discontinuities that degrade control stability. The proposed delay-resilient RL framework, combining LSTM state estimation with residual RL, reduces tracking error by 40% and chattering by 60% compared to baselines on Franka Panda robots.
Stochastic communication delays in teleoperation introduce signal discontinuities that undermine control stability and degrade control performance. Consequently, the conventional reinforcement learning (RL) methods struggle with the delayed observations due to the delay-induced observations, leading to high-frequency chattering. To address this, we propose a hybrid control framework, delay-resilient RL, integrating a state estimator utilizing Long Short-Term Memory (LSTM) with a residual RL policy, which is resilient to stochastic delays. The LSTM reconstructs smooth, continuous state estimates from delayed observations, enabling the RL agent to learn a residual torque compensation policy that balances tracking accuracy with velocity smoothness. Experimental validation on Franka Panda robots demonstrates that our approach significantly outperforms the state-of-the-art baselines, ensuring robust and stable teleoperation even under high-variance stochastic delays.