Dataset Distillation for Offline Reinforcement Learning
This addresses the challenge of obtaining quality datasets for offline reinforcement learning, which is incremental as it applies an existing technique (dataset distillation) to a specific domain.
The paper tackles the problem of offline reinforcement learning by using dataset distillation to synthesize a better dataset, resulting in a model trained on this distilled dataset achieving similar performance to models trained on the full dataset or with percentile behavioral cloning.
Offline reinforcement learning often requires a quality dataset that we can train a policy on. However, in many situations, it is not possible to get such a dataset, nor is it easy to train a policy to perform well in the actual environment given the offline data. We propose using data distillation to train and distill a better dataset which can then be used for training a better policy model. We show that our method is able to synthesize a dataset where a model trained on it achieves similar performance to a model trained on the full dataset or a model trained using percentile behavioral cloning. Our project site is available at https://datasetdistillation4rl.github.io . We also provide our implementation at https://github.com/ggflow123/DDRL .