Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning
This addresses the generalization issue in deep metric learning for image retrieval, offering a simple, incremental improvement applicable to existing proxy-based losses.
The paper tackles the problem of overfitting to seen classes in deep metric learning by proposing Proxy Synthesis, a regularizer that uses synthetic classes to improve generalization to unseen classes, achieving state-of-the-art performance on four image retrieval benchmarks.
One of the main purposes of deep metric learning is to construct an embedding space that has well-generalized embeddings on both seen (training) classes and unseen (test) classes. Most existing works have tried to achieve this using different types of metric objectives and hard sample mining strategies with given training data. However, learning with only the training data can be overfitted to the seen classes, leading to the lack of generalization capability on unseen classes. To address this problem, we propose a simple regularizer called Proxy Synthesis that exploits synthetic classes for stronger generalization in deep metric learning. The proposed method generates synthetic embeddings and proxies that work as synthetic classes, and they mimic unseen classes when computing proxy-based losses. Proxy Synthesis derives an embedding space considering class relations and smooth decision boundaries for robustness on unseen classes. Our method is applicable to any proxy-based losses, including softmax and its variants. Extensive experiments on four famous benchmarks in image retrieval tasks demonstrate that Proxy Synthesis significantly boosts the performance of proxy-based losses and achieves state-of-the-art performance.