SIRIUS-SQL: Anchoring Multi-Candidate Text-to-SQL in Execution Feedback
For practitioners needing reliable Text-to-SQL on complex schemas, SIRIUS-SQL provides a more robust multi-candidate framework that outperforms prior systems.
SIRIUS-SQL addresses three weaknesses in multi-candidate Text-to-SQL pipelines—redundant candidates, generic error correction, and single-angle selection—by introducing difficulty-smoothing RL training, execution-grounded targeted repair, and a confidence-gated hybrid selector, achieving 75.88% on BIRD dev and 91.20% on SPIDER test.
Text-to-SQL on complex schemas is unreliable on a single pass, so recent systems generate multiple SQL candidates and let voting filter out errors. Yet voting alone is not enough, because the multi-candidate recipe has three coupled weaknesses: 1) sampling more from a single generator produces increasingly redundant candidates, 2) existing pipelines apply one generic correction to every non-clean execution result, while runtime errors, timeouts, and empty results each indicate a different distance from correctness, and 3) existing selectors rely on a single angle such as result-majority voting or pairwise SQL comparison, missing what other angles would have caught. We present SIRIUS-SQL, which addresses all three weaknesses. A difficulty-smoothing RL recipe trains SIRIUS-32B to generate diverse executable SQL candidates, paired with a generalist LLM that fills in gaps left by the specialist. An execution-grounded lifecycle classifies each outcome and applies targeted repair before candidates re-enter the pool. A confidence-gated hybrid selector combines execution-result agreement with pairwise SQL-form judgment, escalating only near-tied cases to a deterministic structural check. SIRIUS-SQL reaches 75.88% on BIRD dev and 91.20% on SPIDER test. Two of three generalist pairings surpass Agentar-Scale-SQL, the strongest published multi-candidate system on BIRD dev.