Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Models
This addresses a critical issue for downstream NLP tasks like NL2SQL and table QA in enterprises and sciences, though it is incremental as it builds on existing LLM approaches.
The paper tackled the problem of expanding abbreviated column names in tabular data, such as 'esal' to 'employee salary', by introducing Columbo, an LLM-based solution that outperformed the prior state-of-the-art method by 4-29% across five datasets.
Expanding the abbreviated column names of tables, such as "esal" to "employee salary", is critical for many downstream NLP tasks for tabular data, such as NL2SQL, table QA, and keyword search. This problem arises in enterprises, domain sciences, government agencies, and more. In this paper, we make three contributions that significantly advance the state of the art. First, we show that the synthetic public data used by prior work has major limitations, and we introduce four new datasets in enterprise/science domains, with real-world abbreviations. Second, we show that accuracy measures used by prior work seriously undercount correct expansions, and we propose new synonym-aware measures that capture accuracy much more accurately. Finally, we develop Columbo, a powerful LLM-based solution that exploits context, rules, chain-of-thought reasoning, and token-level analysis. Extensive experiments show that Columbo significantly outperforms NameGuess, the current most advanced solution, by 4-29%, over five datasets. Columbo has been used in production on EDI, a major data lake for environmental sciences.