Clinical Notes Reveal Physician Fatigue
This work addresses physician fatigue detection in healthcare settings, revealing potential biases and implications for LLM-generated text, though it is incremental in applying existing methods to new data.
The researchers developed a model to identify clinical notes written by fatigued physicians using emergency room visit data, finding that it accurately flags notes from high-workload settings and correlates with 18% lower testing yield for heart attacks per standard deviation increase in predicted fatigue. They also discovered that notes about Black and Hispanic patients have 12% and 21% higher predicted fatigue than those about White patients, and that LLM-generated notes show 17% higher predicted fatigue than real physician notes.
Physicians write notes about patients. In doing so, they reveal much about themselves. Using data from 129,228 emergency room visits, we train a model to identify notes written by fatigued physicians -- those who worked 5 or more of the prior 7 days. In a hold-out set, the model accurately identifies notes written by these high-workload physicians, and also flags notes written in other high-fatigue settings: on overnight shifts, and after high patient volumes. Model predictions also correlate with worse decision-making on at least one important metric: yield of testing for heart attack is 18% lower with each standard deviation increase in model-predicted fatigue. Finally, the model indicates that notes written about Black and Hispanic patients have 12% and 21% higher predicted fatigue than Whites -- larger than overnight vs. daytime differences. These results have an important implication for large language models (LLMs). Our model indicates that fatigued doctors write more predictable notes. Perhaps unsurprisingly, because word prediction is the core of how LLMs work, we find that LLM-written notes have 17% higher predicted fatigue than real physicians' notes. This indicates that LLMs may introduce distortions in generated text that are not yet fully understood.