LLM-IE: A Python Package for Generative Information Extraction with Large Language Models
This provides a practical tool for the biomedical NLP community to build information extraction pipelines, but it is incremental as it adapts existing LLM methods into a software package.
The authors tackled the lack of dedicated software for generative information extraction with large language models in biomedicine by developing LLM-IE, a Python package that achieved the best performance on i2b2 datasets using sentence-based prompting, though with longer inference times.
Objectives: Despite the recent adoption of large language models (LLMs) for biomedical information extraction, challenges in prompt engineering and algorithms persist, with no dedicated software available. To address this, we developed LLM-IE: a Python package for building complete information extraction pipelines. Our key innovation is an interactive LLM agent to support schema definition and prompt design. Materials and Methods: The LLM-IE supports named entity recognition, entity attribute extraction, and relation extraction tasks. We benchmarked on the i2b2 datasets and conducted a system evaluation. Results: The sentence-based prompting algorithm resulted in the best performance while requiring a longer inference time. System evaluation provided intuitive visualization. Discussion: LLM-IE was designed from practical NLP experience in healthcare and has been adopted in internal projects. It should hold great value to the biomedical NLP community. Conclusion: We developed a Python package, LLM-IE, that provides building blocks for robust information extraction pipeline construction.