A Multilingual Translator to SQL with Database Schema Pruning to Improve Self-Attention
This work addresses efficiency and accuracy issues in multilingual text-to-SQL systems for database querying, though it is incremental as it builds on existing methods with specific optimizations.
The paper tackles the challenge of long text sequences in transformers for natural language to SQL translation by introducing database schema pruning and a multilingual approach, increasing exact set match accuracy from 0.718 to 0.736 on a validation dataset.
Long sequences of text are challenging in the context of transformers, due to quadratic memory increase in the self-attention mechanism. As this issue directly affects the translation from natural language to SQL queries (as techniques usually take as input a concatenated text with the question and the database schema), we present techniques that allow long text sequences to be handled by transformers with up to 512 input tokens. We propose a training process with database schema pruning (removal of tables and columns names that are useless for the query of interest). In addition, we used a multilingual approach with the mT5-large model fine-tuned with a data-augmented Spider dataset in four languages simultaneously: English, Portuguese, Spanish, and French. Our proposed technique used the Spider dataset and increased the exact set match accuracy results from 0.718 to 0.736 in a validation dataset (Dev). Source code, evaluations, and checkpoints are available at: \underline{https://github.com/C4AI/gap-text2sql}.