Large Multimodal Model based Standardisation of Pathology Reports with Confidence and their Prognostic Significance
This work addresses the challenge of standardizing pathology reports for clinicians and researchers, though it is incremental as it builds on existing LMM methods with added confidence estimation.
The authors tackled the problem of extracting structured information from unstructured pathology reports using large multimodal models, achieving a method that assigns confidence scores to extracted fields and demonstrating prognostic significance for patient stratification.
Pathology reports are rich in clinical and pathological details but are often presented in free-text format. The unstructured nature of these reports presents a significant challenge limiting the accessibility of their content. In this work, we present a practical approach based on the use of large multimodal models (LMMs) for automatically extracting information from scanned images of pathology reports with the goal of generating a standardised report specifying the value of different fields along with estimated confidence about the accuracy of the extracted fields. The proposed approach overcomes limitations of existing methods which do not assign confidence scores to extracted fields limiting their practical use. The proposed framework uses two stages of prompting a Large Multimodal Model (LMM) for information extraction and validation. The framework generalises to textual reports from multiple medical centres as well as scanned images of legacy pathology reports. We show that the estimated confidence is an effective indicator of the accuracy of the extracted information that can be used to select only accurately extracted fields. We also show the prognostic significance of structured and unstructured data from pathology reports and show that the automatically extracted field values significant prognostic value for patient stratification. The framework is available for evaluation via the URL: https://labieb.dcs.warwick.ac.uk/.