DIAGNOSE: Avoiding Out-of-distribution Data using Submodular Information Measures
This addresses the challenge of reducing labeling costs and improving model reliability in medical imaging by mitigating OOD data issues, though it appears incremental as it builds on active learning with a novel combination of submodular measures.
The paper tackles the problem of avoiding out-of-distribution (OOD) data in active learning for medical imaging, where labeling is costly, by proposing Diagnose, a framework that jointly models similarity and dissimilarity to select in-distribution samples; experiments show it outperforms state-of-the-art methods across multiple medical imaging domains.
Avoiding out-of-distribution (OOD) data is critical for training supervised machine learning models in the medical imaging domain. Furthermore, obtaining labeled medical data is difficult and expensive since it requires expert annotators like doctors, radiologists, etc. Active learning (AL) is a well-known method to mitigate labeling costs by selecting the most diverse or uncertain samples. However, current AL methods do not work well in the medical imaging domain with OOD data. We propose Diagnose (avoiDing out-of-dIstribution dAta usinG submodular iNfOrmation meaSurEs), a novel active learning framework that can jointly model similarity and dissimilarity, which is crucial in mining in-distribution data and avoiding OOD data at the same time. Particularly, we use a small number of data points as exemplars that represent a query set of in-distribution data points and a private set of OOD data points. We illustrate the generalizability of our framework by evaluating it on a wide variety of real-world OOD scenarios. Our experiments verify the superiority of Diagnose over the state-of-the-art AL methods across multiple domains of medical imaging.