Deep Neural Networks as Complex Networks
This provides a physically grounded, complementary approach to explain DNN behavior for researchers in machine learning and network theory, though it is incremental as it applies existing theory to DNNs.
The authors tackled the problem of analyzing deep neural networks (DNNs) by representing them as directed weighted graphs using Complex Network Theory (CNT), introducing metrics to study DNNs as dynamical systems and showing that these metrics can discriminate between low and high performing networks.
Deep Neural Networks are, from a physical perspective, graphs whose `links` and `vertices` iteratively process data and solve tasks sub-optimally. We use Complex Network Theory (CNT) to represents Deep Neural Networks (DNNs) as directed weighted graphs: within this framework, we introduce metrics to study DNNs as dynamical systems, with a granularity that spans from weights to layers, including neurons. CNT discriminates networks that differ in the number of parameters and neurons, the type of hidden layers and activations, and the objective task. We further show that our metrics discriminate low vs. high performing networks. CNT is a comprehensive method to reason about DNNs and a complementary approach to explain a model's behavior that is physically grounded to networks theory and goes beyond the well-studied input-output relation.