IRDS: Interpretable RLVR Data Selection via Verifier-Coupled Sparse Autoencoder Coverage
For practitioners of RLVR-based LLM reasoning, IRDS provides an interpretable and efficient data selection method that outperforms existing approaches.
IRDS selects RLVR training instances using a sparse autoencoder cluster basis with a verifier-coupled coverage objective, improving math reasoning accuracy by up to +4.0 pp over baselines while being an order of magnitude cheaper than trajectory-based methods.
Reinforcement learning with verifiable rewards (RLVR) has become a key technique for en- hancing LLM reasoning, yet its data ineffi- ciency remains a major bottleneck. Existing methods address this problem only partially, each missing at least one of subset-level cov- erage, verifier signal use, or interpretability. To address this gap, we present IRDS (Inter- pretable RLVR Data Selection), which selects RLVR training instances on a sparse autoen- coder (SAE) cluster basis so the selection itself is auditable on recognizable problem motifs. To select instances the model both fails on and can still learn from, we introduce a verifier- coupled coverage objective on the SAE basis and solve it by greedy log-determinant max- imization. Experiments on three instruction- tuned models and six math reasoning bench- marks show that IRDS achieves the highest overall accuracy, exceeding the strongest base- line by +3.9/+4.0 pp on the two Qwen models and by +0.5 pp on Llama-3.1-8B, while run- ning an order of magnitude cheaper than the trajectory-based baseline.