Training Overhead Ratio: A Practical Reliability Metric for Large Language Model Training Systems
This work addresses the high training costs due to failures in LLM systems, providing a practical tool for users, though it is incremental as it focuses on metric development rather than solving the underlying reliability issues.
The authors tackled the problem of frequent failures in large language model training systems by introducing a novel reliability metric called Training Overhead Ratio (TOR), which helps estimate actual training time and identifies key factors for enhancing reliability.
Large Language Models (LLMs) are revolutionizing the AI industry with their superior capabilities. Training these models requires large-scale GPU clusters and significant computing time, leading to frequent failures that significantly increase training costs. Despite its significance, this field lacks a metric for evaluating reliability. In this work, we introduce a novel reliability metric called \emph{Training Overhead Ratio} (TOR) to evaluate the reliability of fault-tolerant LLM training systems. TOR is defined as the ratio of optimal training time to the observed training time of a system, serving as a practical tool for users to estimate the actual time required to train an LLM on a given system. Furthermore, our investigation identifies the key factor for enhancing reliability and present TOR equations for various types of failures encountered in practice.