Two-Stage Prompt Optimization for Few-Shot Relation Extraction: From Reasoning-Guided Search to Gradient-Guided Refinement
For practitioners of few-shot relation extraction with small LMs, this work offers a practical method to improve prompt optimization, though the gains are incremental.
The paper tackles automatic prompt optimization for few-shot relation extraction with smaller language models, proposing a two-stage framework combining reasoning-based and gradient-based optimization. It achieves state-of-the-art performance on FS-TACRED with Qwen3-4B and competitive results on FS-FewRel.
Automatic prompt optimization is still underexplored for episodic few-shot relation extraction with smaller language models. We propose a two-stage framework that combines reasoning-based prompt optimization with gradient-based prompt optimization. The first stage can use any reasoning-based optimizer to make broadprompt improvements in natural language. The second stage applies our GradPO, which uses loss and gradient signals to identify high-impact prompt spans and refine them with local edits. Experiments on FS-TACRED and FS-FewRel show that local refinement usually improves prompts found by the first stage, and GradPO is the most consistent refiner. Our framework achieves state-of-the-art performance on FS-TACRED with Qwen3-4B and remains competitive on FS-FewRel.