Geodesic Prototype Matching via Diffusion Maps for Interpretable Fine-Grained Recognition
This work addresses the challenge of interpretable fine-grained recognition for computer vision applications, offering a novel paradigm that improves accuracy and semantic alignment, though it is incremental in advancing prototype-based methods.
The paper tackles the problem of prototype-based interpretable fine-grained recognition by addressing the limitations of Euclidean distances in capturing true similarity on nonlinear manifolds of deep visual features. The proposed GeoProto framework uses diffusion maps and differentiable Nyström interpolation to anchor similarity in intrinsic geometry, resulting in prototypes that focus on semantically aligned parts and significantly outperform Euclidean prototype networks on CUB-200-2011 and Stanford Cars datasets.
Nonlinear manifolds are widespread in deep visual features, where Euclidean distances often fail to capture true similarity. This limitation becomes particularly severe in prototype-based interpretable fine-grained recognition, where subtle semantic distinctions are essential. To address this challenge, we propose a novel paradigm for prototype-based recognition that anchors similarity within the intrinsic geometry of deep features. Specifically, we distill the latent manifold structure of each class into a diffusion space and introduce a differentiable Nyström interpolation, making the geometry accessible to both unseen samples and learnable prototypes. To ensure efficiency, we employ compact per-class landmark sets with periodic updates. This design keeps the embedding aligned with the evolving backbone, enabling fast and scalable inference. Extensive experiments on the CUB-200-2011 and Stanford Cars datasets show that our GeoProto framework produces prototypes focusing on semantically aligned parts, significantly outperforming Euclidean prototype networks.