ILLUMINER: Instruction-tuned Large Language Models as Few-shot Intent Classifier and Slot Filler
This addresses the problem of data-intensive models for industry applications in natural language understanding, offering a more efficient few-shot approach.
The study tackled intent classification and slot filling by framing them as language generation tasks for instruction-tuned large language models, resulting in outperforming state-of-the-art methods with improvements of 11.1–32.2 percentage points in slot filling and requiring less than 6% of training data for comparable performance.
State-of-the-art intent classification (IC) and slot filling (SF) methods often rely on data-intensive deep learning models, limiting their practicality for industry applications. Large language models on the other hand, particularly instruction-tuned models (Instruct-LLMs), exhibit remarkable zero-shot performance across various natural language tasks. This study evaluates Instruct-LLMs on popular benchmark datasets for IC and SF, emphasizing their capacity to learn from fewer examples. We introduce ILLUMINER, an approach framing IC and SF as language generation tasks for Instruct-LLMs, with a more efficient SF-prompting method compared to prior work. A comprehensive comparison with multiple baselines shows that our approach, using the FLAN-T5 11B model, outperforms the state-of-the-art joint IC+SF method and in-context learning with GPT3.5 (175B), particularly in slot filling by 11.1--32.2 percentage points. Additionally, our in-depth ablation study demonstrates that parameter-efficient fine-tuning requires less than 6% of training data to yield comparable performance with traditional full-weight fine-tuning.