PufferLib: Making Reinforcement Learning Libraries and Environments Play Nice
This solves a practical interoperability problem for reinforcement learning researchers and practitioners, though it is incremental as it builds on existing libraries and environments.
The paper tackles the problem of compatibility issues between reinforcement learning environments and libraries by introducing PufferLib, which provides one-line wrappers and fast vectorization to enable seamless integration and accelerate training across diverse benchmarks.
You have an environment, a model, and a reinforcement learning library that are designed to work together but don't. PufferLib makes them play nice. The library provides one-line environment wrappers that eliminate common compatibility problems and fast vectorization to accelerate training. With PufferLib, you can use familiar libraries like CleanRL and SB3 to scale from classic benchmarks like Atari and Procgen to complex simulators like NetHack and Neural MMO. We release pip packages and prebuilt images with dependencies for dozens of environments. All of our code is free and open-source software under the MIT license, complete with baselines, documentation, and support at pufferai.github.io.