ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
This work addresses the challenge of creating structured datasets for NER tasks, particularly in niche domains, offering a more efficient solution for researchers and practitioners.
The paper tackles the problem of generating high-quality named entity recognition datasets by using self-reflective large language models to produce domain-relevant attributes and entity terms, resulting in significant performance improvements and cost-effectiveness compared to conventional methods.
Although Large Language Models (LLMs) exhibit remarkable adaptability across domains, these models often fall short in structured knowledge extraction tasks such as named entity recognition (NER). This paper explores an innovative, cost-efficient strategy to harness LLMs with modest NER capabilities for producing superior NER datasets. Our approach diverges from the basic class-conditional prompts by instructing LLMs to self-reflect on the specific domain, thereby generating domain-relevant attributes (such as category and emotions for movie reviews), which are utilized for creating attribute-rich training data. Furthermore, we preemptively generate entity terms and then develop NER context data around these entities, effectively bypassing the LLMs' challenges with complex structures. Our experiments across both general and niche domains reveal significant performance enhancements over conventional data generation methods while being more cost-effective than existing alternatives.