SEAL : Interactive Tool for Systematic Error Analysis and Labeling
This addresses the challenge of error analysis for NLP models when explicit labels or visual features are lacking, though it is incremental as it builds on existing methods for error analysis and labeling.
The paper tackles the problem of identifying systematic failures of large language models on tail data or rare groups, which are not obvious in aggregate evaluations, by introducing an interactive tool called SEAL that uses a two-step approach to find high-error data slices and assign human-understandable semantics to them.
With the advent of Transformers, large language models (LLMs) have saturated well-known NLP benchmarks and leaderboards with high aggregate performance. However, many times these models systematically fail on tail data or rare groups not obvious in aggregate evaluation. Identifying such problematic data groups is even more challenging when there are no explicit labels (e.g., ethnicity, gender, etc.) and further compounded for NLP datasets due to the lack of visual features to characterize failure modes (e.g., Asian males, animals indoors, waterbirds on land, etc.). This paper introduces an interactive Systematic Error Analysis and Labeling (\seal) tool that uses a two-step approach to first identify high error slices of data and then, in the second step, introduce methods to give human-understandable semantics to those underperforming slices. We explore a variety of methods for coming up with coherent semantics for the error groups using language models for semantic labeling and a text-to-image model for generating visual features. SEAL toolkit and demo screencast is available at https://huggingface.co/spaces/nazneen/seal.