Enriched Annotations for Tumor Attribute Classification from Pathology Reports with Limited Labeled Data
This work is significant for clinical researchers and healthcare providers by enabling more efficient information extraction from unstructured pathology reports, which is crucial for precision medicine, especially in scenarios with limited data.
This paper addresses the challenge of extracting tumor attributes from pathology reports with limited labeled data. The authors developed an enriched hierarchical annotation scheme and an algorithm called Supervised Line Attention (SLA), which achieved similar or better micro-f1 and macro-f1 scores with half the labeled documents compared to state-of-the-art methods, resulting in a 40% reduction in total annotation time.
Precision medicine has the potential to revolutionize healthcare, but much of the data for patients is locked away in unstructured free-text, limiting research and delivery of effective personalized treatments. Generating large annotated datasets for information extraction from clinical notes is often challenging and expensive due to the high level of expertise needed for high quality annotations. To enable natural language processing for small dataset sizes, we develop a novel enriched hierarchical annotation scheme and algorithm, Supervised Line Attention (SLA), and apply this algorithm to predicting categorical tumor attributes from kidney and colon cancer pathology reports from the University of California San Francisco (UCSF). Whereas previous work only annotated document level labels, we in addition ask the annotators to enrich the traditional label by asking them to also highlight the relevant line or potentially lines for the final label, which leads to a 20% increase of annotation time required per document. With the enriched annotations, we develop a simple and interpretable machine learning algorithm that first predicts the relevant lines in the document and then predicts the tumor attribute. Our results show across the small dataset sizes of 32, 64, 128, and 186 labeled documents per cancer, SLA only requires half the number of labeled documents as state-of-the-art methods to achieve similar or better micro-f1 and macro-f1 scores for the vast majority of comparisons that we made. Accounting for the increased annotation time, this leads to a 40% reduction in total annotation time over the state of the art.