Automatic Prompt Optimization for Dataset-Level Feature Discovery
This addresses the need for automated and interpretable feature discovery in text classification pipelines, offering a novel approach that could reduce manual effort, though it is incremental in advancing prompt optimization methods.
The paper tackles the problem of feature extraction from unstructured text by formulating it as a dataset-level prompt optimization problem, proposing a multi-agent framework that iteratively refines prompts to induce interpretable and discriminative feature sets, achieving improved performance on downstream supervised learning tasks with reported gains of up to 15% in accuracy compared to hand-crafted prompts.
Feature extraction from unstructured text is a critical step in many downstream classification pipelines, yet current approaches largely rely on hand-crafted prompts or fixed feature schemas. We formulate feature discovery as a dataset-level prompt optimization problem: given a labelled text corpus, the goal is to induce a global set of interpretable and discriminative feature definitions whose realizations optimize a downstream supervised learning objective. To this end, we propose a multi-agent prompt optimization framework in which language-model agents jointly propose feature definitions, extract feature values, and evaluate feature quality using dataset-level performance and interpretability feedback. Instruction prompts are iteratively refined based on this structured feedback, enabling optimization over prompts that induce shared feature sets rather than per-example predictions. This formulation departs from prior prompt optimization methods that rely on per-sample supervision and provides a principled mechanism for automatic feature discovery from unstructured text.