Curate-Train-Refine: A Closed-Loop Agentic Framework for Zero Shot Classification
This work addresses the practical deployment challenges of large models for classification tasks, offering a more efficient alternative.
The authors tackled the problem of high inference cost and latency in zero-shot classification by proposing a closed-loop framework where an LLM dynamically generates and refines training data for lightweight classifiers, achieving consistent performance improvements across four benchmarks.
Large language models (LLMs) and high-capacity encoders have advanced zero and few-shot classification, but their inference cost and latency limit practical deployment. We propose training lightweight text classifiers using dynamically generated supervision from an LLM. Our method employs an iterative, agentic loop in which the LLM curates training data, analyzes model successes and failures, and synthesizes targeted examples to address observed errors. This closed-loop generation and evaluation process progressively improves data quality and adapts it to the downstream classifier and task. Across four widely used benchmarks, our approach consistently outperforms standard zero and few-shot baselines. These results indicate that LLMs can serve effectively as data curators, enabling accurate and efficient classification without the operational cost of large-model deployment.