QPyTorch: A Low-Precision Arithmetic Simulation Framework
This provides a tool for researchers in machine learning to study low-precision training without hardware changes, but it is incremental as it builds on existing simulation approaches.
The paper tackles the problem of evaluating low-precision training algorithms by introducing QPyTorch, a simulation framework built in PyTorch that minimizes conversion effort and reduces overhead, enabling large-scale simulations.
Low-precision training reduces computational cost and produces efficient models. Recent research in developing new low-precision training algorithms often relies on simulation to empirically evaluate the statistical effects of quantization while avoiding the substantial overhead of building specific hardware. To support this empirical research, we introduce QPyTorch, a low-precision arithmetic simulation framework. Built natively in PyTorch, QPyTorch provides a convenient interface that minimizes the efforts needed to reliably convert existing codes to study low-precision training. QPyTorch is general, and supports a variety of combinations of precisions, number formats, and rounding options. Additionally, it leverages an efficient fused-kernel approach to reduce simulator overhead, which enables simulation of large-scale, realistic problems. QPyTorch is publicly available at https://github.com/Tiiiger/QPyTorch.