Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
This addresses the issue of poor generalization in Transformers for researchers and practitioners, offering an automated, efficient method without manual heuristics, though it is incremental as it builds on curriculum learning.
The paper tackled the problem of Transformer models' brittle extrapolation on longer or more complex inputs than seen during training by introducing Counter-Example-Driven Curricula (CEDC), which improved robustness with up to 30x greater length extrapolation and 3.75x more computational efficiency than baselines.
Transformer models often exhibit brittle extrapolation, failing on inputs that are longer or structurally more complex than those seen during training. We introduce Counter-Example-Driven Curricula (CEDC), an automated framework that improves model robustness by iteratively focusing on its own failures. At each step, CEDC uses the current model to generate a diverse set of candidate problems, employs a fast, executable verifier to identify incorrect predictions (counter-examples), and then fine-tunes the model on a dataset enriched with these discovered failures. We evaluate CEDC on a suite of algorithmic and natural language tasks, including integer addition, sorting, Dyck-2 language recognition, and three text classification benchmarks. Compared to static training and standard curriculum learning baselines, CEDC achieves up to 30x greater length extrapolation, is 3.75x more computationally efficient than uniform data augmentation, and requires no manual difficulty heuristics. We provide a detailed analysis of the counter-examples, showing how the curriculum naturally adapts to target progressively more complex error modes. Our findings establish verifier-guided, failure-driven learning as a simple, powerful, and efficient paradigm for enhancing the generalization capabilities of Transformer models.