A Large Language Model Pipeline for Breast Cancer Oncology
This work addresses the need for AI-assisted treatment decisions in oncology, particularly for expanding access to quality care in community facilities, but it is incremental as it builds on existing LLM methods with a new pipeline and dataset.
The authors tackled the problem of applying large language models to oncology by fine-tuning OpenAI models on clinical data and guidelines for breast cancer treatment decisions, achieving over 85% accuracy in classifying adjuvant radiation therapy and chemotherapy, and estimating that the model must outperform human oncologists in 8.2% to 13.3% of scenarios to be a better overall solution.
Large language models (LLMs) have demonstrated potential in the innovation of many disciplines. However, how they can best be developed for oncology remains underdeveloped. State-of-the-art OpenAI models were fine-tuned on a clinical dataset and clinical guidelines text corpus for two important cancer treatment factors, adjuvant radiation therapy and chemotherapy, using a novel Langchain prompt engineering pipeline. A high accuracy (0.85+) was achieved in the classification of adjuvant radiation therapy and chemotherapy for breast cancer patients. Furthermore, a confidence interval was formed from observational data on the quality of treatment from human oncologists to estimate the proportion of scenarios in which the model must outperform the original oncologist in its treatment prediction to be a better solution overall as 8.2% to 13.3%. Due to indeterminacy in the outcomes of cancer treatment decisions, future investigation, potentially a clinical trial, would be required to determine if this threshold was met by the models. Nevertheless, with 85% of U.S. cancer patients receiving treatment at local community facilities, these kinds of models could play an important part in expanding access to quality care with outcomes that lie, at minimum, close to a human oncologist.