A Clinical Trial Design Approach to Auditing Language Models in Healthcare Setting
This addresses the need for reliable and efficient auditing of language models in healthcare settings, offering a structured approach to ensure model safety and compliance, though it is incremental as it adapts existing clinical trial methods to a new domain.
The paper tackles the problem of auditing language models in healthcare by proposing a mechanism inspired by clinical trial design, treating the audit as a single blind equivalence trial with subject matter experts as the comparison, and demonstrates its application in a real-world production environment with a large-scale public health network, showing it enables principled sample size and power calculations to minimize required records while maintaining audit integrity and statistical soundness.
We present an audit mechanism for language models, with a focus on models deployed in the healthcare setting. Our proposed mechanism takes inspiration from clinical trial design where we posit the language model audit as a single blind equivalence trial, with the comparison of interest being the subject matter experts. We show that using our proposed method, we can follow principled sample size and power calculations, leading to the requirement of sampling minimum number of records while maintaining the audit integrity and statistical soundness. Finally, we provide a real-world example of the audit used in a production environment in a large-scale public health network.