GECO: Geometrically Consistent Embedding with Lightspeed Inference
This work addresses the need for geometry-aware feature learning in computer vision, offering significant speed and accuracy gains for tasks like semantic correspondence, though it is incremental in enhancing existing methods.
The paper tackles the problem of self-supervised vision models lacking 3D geometry awareness by introducing GECO, which produces geometrically coherent features and achieves state-of-the-art performance with improvements of 6.0%, 6.2%, and 4.1% in PCK on PFPascal, APK, and CUB datasets, while running 98.2% faster at 30 fps.
Recent advances in feature learning have shown that self-supervised vision foundation models can capture semantic correspondences but often lack awareness of underlying 3D geometry. GECO addresses this gap by producing geometrically coherent features that semantically distinguish parts based on geometry (e.g., left/right eyes, front/back legs). We propose a training framework based on optimal transport, enabling supervision beyond keypoints, even under occlusions and disocclusions. With a lightweight architecture, GECO runs at 30 fps, 98.2% faster than prior methods, while achieving state-of-the-art performance on PFPascal, APK, and CUB, improving PCK by 6.0%, 6.2%, and 4.1%, respectively. Finally, we show that PCK alone is insufficient to capture geometric quality and introduce new metrics and insights for more geometry-aware feature learning. Link to project page: https://reginehartwig.github.io/publications/geco/