Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking
This provides a tool for researchers in deep reinforcement learning to accelerate experimentation, though it is incremental as it builds on existing libraries and methods.
The authors tackled the problem of limited flexibility and steep learning curves in deep reinforcement learning libraries by introducing Tonic, a Python library that enables fast prototyping and benchmarking, resulting in a benchmark of baseline agents on 70 continuous-control tasks and the implementation of a novel agent called TD4.
Deep reinforcement learning has been one of the fastest growing fields of machine learning over the past years and numerous libraries have been open sourced to support research. However, most codebases have a steep learning curve or limited flexibility that do not satisfy a need for fast prototyping in fundamental research. This paper introduces Tonic, a Python library allowing researchers to quickly implement new ideas and measure their importance by providing: 1) general-purpose configurable modules 2) several baseline agents: A2C, TRPO, PPO, MPO, DDPG, D4PG, TD3 and SAC built with these modules 3) support for TensorFlow 2 and PyTorch 4) support for continuous-control environments from OpenAI Gym, DeepMind Control Suite and PyBullet 5) scripts to experiment in a reproducible way, plot results, and play with trained agents 6) a benchmark of the provided agents on 70 continuous-control tasks. Evaluation is performed in fair conditions with identical seeds, training and testing loops, while sharing general improvements such as non-terminal timeouts and observation normalization. Finally, to demonstrate how Tonic simplifies experimentation, a novel agent called TD4 is implemented and evaluated.