TorchRL: A data-driven decision-making library for PyTorch
This provides a comprehensive library for researchers and developers using PyTorch for reinforcement learning and control tasks, but it is incremental as it builds upon existing frameworks.
The authors tackled the lack of a native PyTorch library for decision and control tasks by proposing TorchRL, a generalistic control library that introduces TensorDict for streamlined algorithm development, and they demonstrated its computational efficiency through benchmarks.
PyTorch has ascended as a premier machine learning framework, yet it lacks a native and comprehensive library for decision and control tasks suitable for large development teams dealing with complex real-world data and environments. To address this issue, we propose TorchRL, a generalistic control library for PyTorch that provides well-integrated, yet standalone components. We introduce a new and flexible PyTorch primitive, the TensorDict, which facilitates streamlined algorithm development across the many branches of Reinforcement Learning (RL) and control. We provide a detailed description of the building blocks and an extensive overview of the library across domains and tasks. Finally, we experimentally demonstrate its reliability and flexibility and show comparative benchmarks to demonstrate its computational efficiency. TorchRL fosters long-term support and is publicly available on GitHub for greater reproducibility and collaboration within the research community. The code is open-sourced on GitHub.