Robust Offline Reinforcement Learning -- Certify the Confidence Interval
This addresses security concerns in RL for practitioners, though it appears incremental as it builds on existing random smoothing techniques.
The paper tackles the security problem in reinforcement learning by developing an algorithm to certify the robustness of policies offline using random smoothing, with experiments confirming its correctness.
Currently, reinforcement learning (RL), especially deep RL, has received more and more attention in the research area. However, the security of RL has been an obvious problem due to the attack manners becoming mature. In order to defend against such adversarial attacks, several practical approaches are developed, such as adversarial training, data filtering, etc. However, these methods are mostly based on empirical algorithms and experiments, without rigorous theoretical analysis of the robustness of the algorithms. In this paper, we develop an algorithm to certify the robustness of a given policy offline with random smoothing, which could be proven and conducted as efficiently as ones without random smoothing. Experiments on different environments confirm the correctness of our algorithm.