Heated-Up Softmax Embedding
This work addresses metric learning for improved sample embeddings, but it is incremental as it builds on existing softmax-based methods with a novel training strategy.
The paper tackles the problem of learning semantic distances in metric learning by showing that training classifiers with varying softmax temperatures affects feature compactness, and proposes a 'heating-up' strategy that achieves state-of-the-art performance on multiple benchmarks.
Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples. The problem is generally solved by learning an embedding for each sample such that the embeddings of samples of the same category are compact while the embeddings of samples of different categories are spread-out in the feature space. We study the features extracted from the second last layer of a deep neural network based classifier trained with the cross entropy loss on top of the softmax layer. We show that training classifiers with different temperature values of softmax function leads to features with different levels of compactness. Leveraging these insights, we propose a "heating-up" strategy to train a classifier with increasing temperatures, leading the corresponding embeddings to achieve state-of-the-art performance on a variety of metric learning benchmarks.