Can Large Language Models Follow Concept Annotation Guidelines? A Case Study on Scientific and Financial Domains
This work addresses the problem of concept understanding in LLMs for researchers and practitioners, revealing gaps between open-source and proprietary models, but it is incremental as it builds on existing evaluation methods.
The study investigated whether instruction-tuned large language models can follow concept annotation guidelines for sentence labeling tasks, finding that only larger models (70B+ parameters) show limited ability in counterfactual contexts, and proprietary models like GPT-3.5/4 outperform open-source ones, with Falcon-180B-chat often lagging behind Llama-2-70B-chat.
Although large language models (LLMs) exhibit remarkable capacity to leverage in-context demonstrations, it is still unclear to what extent they can learn new concepts or facts from ground-truth labels. To address this question, we examine the capacity of instruction-tuned LLMs to follow in-context concept guidelines for sentence labeling tasks. We design guidelines that present different types of factual and counterfactual concept definitions, which are used as prompts for zero-shot sentence classification tasks. Our results show that although concept definitions consistently help in task performance, only the larger models (with 70B parameters or more) have limited ability to work under counterfactual contexts. Importantly, only proprietary models such as GPT-3.5 and GPT-4 can recognize nonsensical guidelines, which we hypothesize is due to more sophisticated alignment methods. Finally, we find that Falcon-180B-chat is outperformed by Llama-2-70B-chat is most cases, which indicates that careful fine-tuning is more effective than increasing model scale. Altogether, our simple evaluation method reveals significant gaps in concept understanding between the most capable open-source language models and the leading proprietary APIs.