Considerations for health care institutions training large language models on electronic health records
This work addresses practical cost and scalability challenges for healthcare institutions aiming to implement LLMs on EHR data, but it is incremental as it focuses on analysis rather than new methods or results.
The study analyzed dataset sizes, model sizes, and costs to provide a framework for healthcare institutions deciding whether to train large language models from scratch or fine-tune them on electronic health record data within budget constraints.
Large language models (LLMs) like ChatGPT have excited scientists across fields; in medicine, one source of excitement is the potential applications of LLMs trained on electronic health record (EHR) data. But there are tough questions we must first answer if health care institutions are interested in having LLMs trained on their own data; should they train an LLM from scratch or fine-tune it from an open-source model? For healthcare institutions with a predefined budget, what are the biggest LLMs they can afford? In this study, we take steps towards answering these questions with an analysis on dataset sizes, model sizes, and costs for LLM training using EHR data. This analysis provides a framework for thinking about these questions in terms of data scale, compute scale, and training budgets.