GoldenTransformer: A Modular Fault Injection Framework for Transformer Robustness Research
This provides a tool for researchers and practitioners to analyze model robustness and guide dependable system design in real-world LLM applications, though it is incremental as it builds on existing fault injection concepts for transformers.
The authors tackled the underexplored problem of transformer robustness under fault conditions by developing GoldenTransformer, a modular fault injection framework that enables controlled evaluation of Large Language Models through diverse fault classes, such as weight corruption and attention disruptions, with support for reproducibility and visualization.
Transformers have become the foundation for a wide range of state--of--the--art models across natural language processing, computer vision, and other machine learning domains. Despite their widespread deployment, the robustness of these models under fault conditions remains underexplored. We present GoldenTransformer, a modular and extensible fault injection framework designed to evaluate the resiliency of Large Language Models to induced hardware faults. GoldenTransformer offers a unified Python-based platform for injecting diverse classes of faults--such as weight corruption, activation injections, and attention--level disruptions--into pretrained transformer--based models. Inspired by the GoldenEye simulator for DNNs, our framework focuses on the unique challenges of working with large transformer architectures, including considerations such as structural complexity, latent dependencies, and nonuniform layer definitions. GoldenTransformer is built atop PyTorch and HuggingFace Transformers, and it supports experiment reproducibility, metric logging, and visualization out of the box. We detail the technical design and use of GoldenTransformer and demonstrate through several example experiments on classification and generation tasks. By enabling controlled injection of faults at multiple logical and structural points in a transformer, GoldenTransformer offers researchers and practitioners a valuable tool for model robustness analysis and for guiding dependable system design in real-world LLM applications.