ID-Sim: An Identity-Focused Similarity Metric
This addresses the need for better evaluation metrics in identity-focused tasks like personalized image generation, though it appears incremental as it builds on existing similarity metric approaches.
The authors tackled the problem of vision models lacking identity-focused evaluation metrics by proposing ID-Sim, a feed-forward metric that reflects human selective sensitivity, and they evaluated it on a new benchmark showing consistency with human annotations across tasks.
Humans have remarkable selective sensitivity to identities -- easily distinguishing between highly similar identities, even across significantly different contexts such as diverse viewpoints or lighting. Vision models have struggled to match this capability, and progress toward identity-focused tasks such as personalized image generation is slowed by a lack of identity-focused evaluation metrics. To help facilitate progress, we propose ID-Sim, a feed-forward metric designed to faithfully reflect human selective sensitivity. To build ID-Sim, we curate a high-quality training set of images spanning diverse real-world domains, augmented with generative synthetic data that provides controlled, fine-grained identity and contextual variations. We evaluate our metric on a new unified evaluation benchmark for assessing consistency with human annotations across identity-focused recognition, retrieval, and generative tasks.