GIFT: Global stabilisation via Intrinsic Fine Tuning
For practitioners deploying deep RL in real-world control systems requiring stability guarantees, GIFT offers a method to enhance stability without sacrificing performance.
GIFT is a training framework that directly optimizes the global stability of deep RL policies, reducing sensitivity to initial conditions while maintaining task performance. It improves the suitability of these policies for real-world control systems.
Deep reinforcement learning policies achieve strong performance in complex continuous control environments with nonlinear contact forces. However, these policies often produce chaotic state dynamics, with trivially small changes to the initial conditions significantly impacting the long-term behaviour of the control system. This high sensitivity to initial conditions limits the application of Deep RL to real-world control systems where performance and stability guarantees are often required. To address this issue, we propose Global stabilisation via Intrinsic Fine Tuning (GIFT), a general-purpose training framework which directly optimises the global stability of existing high-performing deep RL policies using a custom reward function. We demonstrate that GIFT increase the stability of the control interaction while maintaining comparable task performance, thereby improving the suitability of deep RL policies for real-world control systems.