PyTorch: An Imperative Style, High-Performance Deep Learning Library
This work provides a foundational tool for researchers and practitioners in machine learning by enabling both flexibility and efficiency, though it is incremental in improving upon existing frameworks.
The paper tackles the challenge of combining usability and speed in deep learning frameworks by introducing PyTorch, a library that offers an imperative, Pythonic programming style for easy debugging and integration, while achieving high performance with GPU support, as demonstrated on common benchmarks.
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.