DEFRAG: Deep Euclidean Feature Representations through Adaptation on the Grassmann Manifold
This addresses the challenge of improving feature clustering and separation in deep learning for classification tasks, though it appears incremental as it builds on existing methods with a novel adaptation technique.
The paper tackles the problem of training deep networks to produce Euclidean feature representations with strong clustering properties, achieving state-of-the-art results on standard classification datasets using a smaller network with fewer parameters.
We propose a novel technique for training deep networks with the objective of obtaining feature representations that exist in a Euclidean space and exhibit strong clustering behavior. Our desired features representations have three traits: they can be compared using a standard Euclidian distance metric, samples from the same class are tightly clustered, and samples from different classes are well separated. However, most deep networks do not enforce such feature representations. The DEFRAG training technique consists of two steps: first good feature clustering behavior is encouraged though an auxiliary loss function based on the Silhouette clustering metric. Then the feature space is retracted onto a Grassmann manifold to ensure that the L_2 Norm forms a similarity metric. The DEFRAG technique achieves state of the art results on standard classification datasets using a relatively small network architecture with significantly fewer parameters than many standard networks.