Sparsity-Probe: Analysis tool for Deep Learning Models
This provides a method for researchers and practitioners to analyze deep learning architectures more efficiently, though it is incremental as it builds on existing principles.
The authors tackled the problem of analyzing deep learning models by introducing the Sparsity Probe, a tool that quantifies geometrical features of intermediate layer representations using training data, enabling performance measurement and layer diagnostics without test sets.
We propose a probe for the analysis of deep learning architectures that is based on machine learning and approximation theoretical principles. Given a deep learning architecture and a training set, during or after training, the Sparsity Probe allows to analyze the performance of intermediate layers by quantifying the geometrical features of representations of the training set. We show how the Sparsity Probe enables measuring the contribution of adding depth to a given architecture, to detect under-performing layers, etc., all this without any auxiliary test data set.