Critical appraisal of artificial intelligence for rare-event recognition: principles and pharmacovigilance case studies
This work addresses the problem of ensuring reliable AI performance in high-stakes applications like pharmacovigilance, where rare events are critical but often misjudged, though it is incremental by building on existing evaluation principles.
The paper tackles the challenge of evaluating AI models for rare-event recognition, where low prevalence can mislead performance metrics, and proposes a framework including structured case-level examination and a checklist for critical appraisal, applied to pharmacovigilance case studies.
Many high-stakes AI applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative LLMs constrained for classification. We outline key considerations for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a comprehensive checklist to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports; duplicate detection combining machine learning with probabilistic record linkage; and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets - and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce and error costs may be asymmetric.