LLM Empowered Prototype Learning for Zero and Few-Shot Tasks on Tabular Data
This addresses the problem of limited data availability in tabular learning for researchers and practitioners, though it is incremental as it builds on existing LLM capabilities.
The paper tackles the challenge of using large language models (LLMs) for zero-shot and few-shot learning on tabular data by proposing a framework that queries LLMs to generate feature values from task descriptions, enabling training-free prototype estimation. The method achieves effective results in experiments for these scenarios.
Recent breakthroughs in large language models (LLMs) have opened the door to in-depth investigation of their potential in tabular data modeling. However, effectively utilizing advanced LLMs in few-shot and even zero-shot scenarios is still challenging. To this end, we propose a novel LLM-based prototype estimation framework for tabular learning. Our key idea is to query the LLM to generate feature values based example-free prompt, which solely relies on task and feature descriptions. With the feature values generated by LLM, we can build a zero-shot prototype in a training-free manner, which can be further enhanced by fusing few-shot samples, avoiding training a classifier or finetuning the LLMs. Thanks to the example-free prompt and prototype estimation, ours bypasses the constraints brought by the example-based prompt, providing a scalable and robust framework. Extensive experiments demonstrate the effectiveness of ours in zero and few-shot tabular learning.