Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notes
This work addresses the challenge of privacy restrictions in healthcare AI by enabling publicly shareable models, though it is incremental in using synthetic data as a substitute.
The paper tackled the problem of limited access to real clinical notes for training clinical large language models by creating synthetic clinical notes from public biomedical literature and training a model called Asclepius, which demonstrated high performance comparable to models trained on real notes in evaluations by GPT-4 and medical professionals.
The development of large language models tailored for handling patients' clinical notes is often hindered by the limited accessibility and usability of these notes due to strict privacy regulations. To address these challenges, we first create synthetic large-scale clinical notes using publicly available case reports extracted from biomedical literature. We then use these synthetic notes to train our specialized clinical large language model, Asclepius. While Asclepius is trained on synthetic data, we assess its potential performance in real-world applications by evaluating it using real clinical notes. We benchmark Asclepius against several other large language models, including GPT-3.5-turbo and other open-source alternatives. To further validate our approach using synthetic notes, we also compare Asclepius with its variants trained on real clinical notes. Our findings convincingly demonstrate that synthetic clinical notes can serve as viable substitutes for real ones when constructing high-performing clinical language models. This conclusion is supported by detailed evaluations conducted by both GPT-4 and medical professionals. All resources including weights, codes, and data used in the development of Asclepius are made publicly accessible for future research. (https://github.com/starmpcc/Asclepius)