PHEONA: An Evaluation Framework for Large Language Model-based Approaches to Computational Phenotyping
This work addresses the problem of resource-intensive computational phenotyping for biomedical researchers, but it is incremental as it focuses on evaluation rather than a new method.
The authors tackled the challenge of computational phenotyping by developing PHEONA, an evaluation framework for Large Language Model-based approaches, and demonstrated it on concept classification for Acute Respiratory Failure, achieving high classification accuracy.
Computational phenotyping is essential for biomedical research but often requires significant time and resources, especially since traditional methods typically involve extensive manual data review. While machine learning and natural language processing advancements have helped, further improvements are needed. Few studies have explored using Large Language Models (LLMs) for these tasks despite known advantages of LLMs for text-based tasks. To facilitate further research in this area, we developed an evaluation framework, Evaluation of PHEnotyping for Observational Health Data (PHEONA), that outlines context-specific considerations. We applied and demonstrated PHEONA on concept classification, a specific task within a broader phenotyping process for Acute Respiratory Failure (ARF) respiratory support therapies. From the sample concepts tested, we achieved high classification accuracy, suggesting the potential for LLM-based methods to improve computational phenotyping processes.