An Adversarial Objective for Scalable Exploration
This work addresses scalability challenges in exploration for robotics, offering incremental improvements over existing model-based methods.
The paper tackles the scalability issues of model-based curiosity methods in robotics by introducing an adversarial curiosity approach that uses a discriminator network to guide exploration, resulting in improved sample efficiency and prediction performance in simulated environments and a robotic manipulation task.
Model-based curiosity combines active learning approaches to optimal sampling with the information gain based incentives for exploration presented in the curiosity literature. Existing model-based curiosity methods look to approximate prediction uncertainty with approaches which struggle to scale to many prediction-planning pipelines used in robotics tasks. We address these scalability issues with an adversarial curiosity method minimizing a score given by a discriminator network. This discriminator is optimized jointly with a prediction model and enables our active learning approach to sample sequences of observations and actions which result in predictions considered the least realistic by the discriminator. We demonstrate progressively increasing advantages as compute is restricted of our adversarial curiosity approach over leading model-based exploration strategies in simulated environments. We further demonstrate the ability of our adversarial curiosity method to scale to a robotic manipulation prediction-planning pipeline where we improve sample efficiency and prediction performance for a domain transfer problem.