PHOEBI: An Open-World Benchmark for Bacterial Identification in Phase-Contrast Microscopy
For microbiologists and computer vision researchers, this benchmark exposes a fundamental open-world recognition failure in current models and provides a solution that generalizes to unseen polymicrobial mixtures.
PHOEBI introduces a benchmark for multi-label bacterial identification from phase-contrast microscopy, revealing that gradient-trained aggregators systematically fail on unseen species combinations (dropping 0.39-0.57 F1), while proposed anchor-based decoders achieve higher F1 on held-out combinations than on in-distribution validation.
Optical microscopy enables rapid, label-free imaging of live bacteria and is the standard instrument for species identification across clinical, environmental, and industrial microbiology. Yet field samples are routinely polymicrobial and may contain organisms that were never seen during system training, and no computer-vision benchmark tests multi-label species identification from phase-contrast microscopy (PCM) of such mixtures. We introduce Phase-contrast Optical bEnchmark for Bacterial Identification ($\textbf{PHOEBI}$), a wet-lab-prepared dataset of $120{,}000$ PCM images covering $40$ combinations of six rod-shaped species, paired with a leave-combinations-out (LCO) evaluation protocol that holds out entire species combinations to mirror the practical scenario of a model trained on catalogued mixtures that must generalise to unseen ones. On LCO, every gradient-trained per-image aggregator we test drops $0.39$ to $0.57$ F1 from the in-distribution to the held-out split, a systematic open-world recognition failure in the aggregator, not the visual representation. A linear probe of thirteen different encoders over the same features spreads only about six percentage points of F1 across general-purpose and biomedical pretraining objectives, confirming the representation is sound. We propose three lightweight $\textit{anchor-based}$ decoders that capture per-species presence geometrically over a shared frozen tile-feature pool, scoring $\textit{higher}$ on held-out combinations than on in-distribution validation.