EdgeLM: Edge Demonstrations for Language Models' Table Understanding
This work offers an incremental improvement in demonstration selection for LLMs, specifically benefiting users performing table-centric prediction tasks by improving accuracy without retraining.
This paper addresses the challenge of demonstration selection for large language models (LLMs) in table-centric prediction tasks. The authors propose EdgeLM, a retrieval framework that selects "edge evidence" demonstrations, which are both relevant to the query and informative about decision boundaries. EdgeLM consistently achieves the best or near-best performance across five data wrangling tasks, fifteen datasets, and five different LLMs.
Large language models (LLMs) perform table-centric prediction through in-context learning, making demonstration selection critical to performance. Existing retrieval methods prioritize similarity to the query, but similar demonstrations often reinforce the model's likely prediction rather than reveal the distinctions needed for difficult decisions. We propose EdgeLM, a retrieval framework that instead selects edge evidence, demonstrations that are both relevant to the query and informative about the decision boundary. EdgeLM retrieves two complementary forms of edge evidence by selecting data edges, nearby examples with different ground-truth labels, and model edges, similar examples previously misclassified by the deployed model. EdgeLM requires neither model retraining nor task-specific engineering. Across five data wrangling tasks, fifteen datasets, and five open-weight and proprietary LLMs, EdgeLM consistently achieves the best or near-best performance in every setting, while ablations show that the two forms of edge evidence provide complementary benefits. Our code and datasets are publicly available at https://github.com/soroushomidvar/EdgeLM.