Connecting Sphere Manifolds Hierarchically for Regularization
This addresses classification accuracy for tasks with hierarchical class structures, such as fine-grained image recognition, and is incremental as it builds on existing neural network architectures.
The paper tackles classification with hierarchical classes by forcing each class's classifier to belong to a sphere manifold centered on its super-class's classifier, connecting these manifolds hierarchically. This regularization improves performance of ResNet and DenseNet on datasets like CIFAR100 and Tiny-ImageNet, with concrete gains reported.
This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).