Towards Differentially Private Reinforcement Learning with General Function Approximation
For researchers in privacy-preserving RL, this work extends differential privacy guarantees to general function approximation, a significant step beyond tabular and linear settings, though the regret bound matches existing linear results.
This paper provides the first theoretical guarantees for differentially private online reinforcement learning with general function approximation, achieving regret scaling as Õ(K^{3/5}) in the model-free setting, matching the linear case state of the art. It also establishes the first regret bound for online RL with batch update based on coverability and identifies gaps in prior linear function approximation results.
We present the first theoretical guarantees for differentially private online reinforcement learning (RL) with general function approximation, extending beyond prior work restricted to tabular and linear settings. Our approach combines a batched policy update scheme with the exponential mechanism, together with a novel regret analysis. We show that, even under general function approximation, the regret in the model-free setting under differential privacy matches the state of the art for the linear case, scaling as $\widetilde{O}(K^{3/5})$, where $K$ denotes the number of episodes. As an important by-product, we also establish the first regret bound for online RL with batch update that depends on the standard complexity measure of coverability, complementing existing results based on a newly introduced Eluder-Condition class. In addition, we uncover fundamental gaps in recent results for private RL with linear function approximation, thereby clarifying its landscape.