Curvature-based Comparison of Two Neural Networks
This work addresses the need for better understanding of neural network internal representations, but it is incremental as it builds on existing geometric approaches without demonstrating broad performance gains.
The paper tackled the problem of comparing two deep neural networks by analyzing the manifolds of activation vectors in each fully connected layer, using Riemann and sectional curvature as criteria to reveal similarities and differences in feature extraction and intrinsic mechanisms.
In this paper we show the similarities and differences of two deep neural networks by comparing the manifolds composed of activation vectors in each fully connected layer of them. The main contribution of this paper includes 1) a new data generating algorithm which is crucial for determining the dimension of manifolds; 2) a systematic strategy to compare manifolds. Especially, we take Riemann curvature and sectional curvature as part of criterion, which can reflect the intrinsic geometric properties of manifolds. Some interesting results and phenomenon are given, which help in specifying the similarities and differences between the features extracted by two networks and demystifying the intrinsic mechanism of deep neural networks.