Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
This addresses the challenge of poor performance in offline RL applications with limited data, though it appears incremental as it builds on existing regularization methods.
The paper tackles the problem of offline reinforcement learning algorithms overfitting on small datasets, introducing Sparse-Reg, a sparsity-based regularization technique that outperforms state-of-the-art baselines in continuous control.
In this paper, we investigate the use of small datasets in the context of offline reinforcement learning (RL). While many common offline RL benchmarks employ datasets with over a million data points, many offline RL applications rely on considerably smaller datasets. We show that offline RL algorithms can overfit on small datasets, resulting in poor performance. To address this challenge, we introduce "Sparse-Reg": a regularization technique based on sparsity to mitigate overfitting in offline reinforcement learning, enabling effective learning in limited data settings and outperforming state-of-the-art baselines in continuous control.