Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality
This addresses the problem of optimizing machine learning software quality for developers by showing that non-default bindings can reduce time costs without sacrificing accuracy, though it is incremental as it builds on existing binding technologies.
The paper studied the impact of using TensorFlow and PyTorch bindings in C#, Rust, Python, and JavaScript on software quality, focusing on correctness and time cost across five deep learning models. It found that models can be trained in one binding and used for inference in another without accuracy loss, and non-default bindings can improve time cost compared to default Python bindings while maintaining correctness.
Bindings for machine learning frameworks (such as TensorFlow and PyTorch) allow developers to integrate a framework's functionality using a programming language different from the framework's default language (usually Python). In this paper, we study the impact of using TensorFlow and PyTorch bindings in C#, Rust, Python and JavaScript on the software quality in terms of correctness (training and test accuracy) and time cost (training and inference time) when training and performing inference on five widely used deep learning models. Our experiments show that a model can be trained in one binding and used for inference in another binding for the same framework without losing accuracy. Our study is the first to show that using a non-default binding can help improve machine learning software quality from the time cost perspective compared to the default Python binding while still achieving the same level of correctness.