TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications
This work addresses the need for understanding application resilience to enable error-resilience techniques in safety-critical ML systems, but it is incremental as it builds on prior fault injection methods.
The authors tackled the problem of assessing the reliability of machine learning systems in safety-critical domains by developing TensorFI, a flexible fault injection framework for TensorFlow applications, and used it to evaluate the resilience of 12 ML programs, including DNNs for autonomous vehicles.
As machine learning (ML) has seen increasing adoption in safety-critical domains (e.g., autonomous vehicles), the reliability of ML systems has also grown in importance. While prior studies have proposed techniques to enable efficient error-resilience techniques (e.g., selective instruction duplication), a fundamental requirement for realizing these techniques is a detailed understanding of the application's resilience. In this work, we present TensorFI, a high-level fault injection (FI) framework for TensorFlow-based applications. TensorFI is able to inject both hardware and software faults in general TensorFlow programs. TensorFI is a configurable FI tool that is flexible, easy to use, and portable. It can be integrated into existing TensorFlow programs to assess their resilience for different fault types (e.g., faults in particular operators). We use TensorFI to evaluate the resilience of 12 ML programs, including DNNs used in the autonomous vehicle domain. Our tool is publicly available at https://github.com/DependableSystemsLab/TensorFI.