Boosting Network Weight Separability via Feed-Backward Reconstruction
This work addresses performance enhancement in neural networks for visual recognition, though it appears incremental as it builds on existing concepts of weight separability.
The paper tackles the problem of improving neural network performance by boosting weight separability, proposing a new evaluation metric based on semi-orthogonality and Frobenius distance, and a feed-backward reconstruction loss method. Experimental results on image classification and face recognition show that this approach universally improves performance across various visual recognition tasks.
This paper proposes a new evaluation metric and boosting method for weight separability in neural network design. In contrast to general visual recognition methods designed to encourage both intra-class compactness and inter-class separability of latent features, we focus on estimating linear independence of column vectors in weight matrix and improving the separability of weight vectors. To this end, we propose an evaluation metric for weight separability based on semi-orthogonality of a matrix and Frobenius distance, and the feed-backward reconstruction loss which explicitly encourages weight separability between the column vectors in the weight matrix. The experimental results on image classification and face recognition demonstrate that the weight separability boosting via minimization of feed-backward reconstruction loss can improve the visual recognition performance, hence universally boosting the performance on various visual recognition tasks.