Debugging Machine Learning Pipelines
This addresses the time-consuming and error-prone task of debugging ML pipelines for practitioners, though it appears incremental as it builds on existing methods for root cause analysis.
The paper tackles the problem of debugging failures in machine learning pipelines by proposing an approach that uses iteration and provenance to automatically infer root causes and provide explanations. The result is evaluated for cost, precision, and recall compared to state-of-the-art methods, with source code and data made available for reproducibility.
Machine learning tasks entail the use of complex computational pipelines to reach quantitative and qualitative conclusions. If some of the activities in a pipeline produce erroneous or uninformative outputs, the pipeline may fail or produce incorrect results. Inferring the root cause of failures and unexpected behavior is challenging, usually requiring much human thought, and is both time-consuming and error-prone. We propose a new approach that makes use of iteration and provenance to automatically infer the root causes and derive succinct explanations of failures. Through a detailed experimental evaluation, we assess the cost, precision, and recall of our approach compared to the state of the art. Our source code and experimental data will be available for reproducibility and enhancement.