Assessing Intelligence in Artificial Neural Networks
This work addresses the need for better metrics to evaluate neural network efficiency and robustness, particularly for researchers and practitioners designing compact and reliable models, though it is incremental as it builds on existing architectures and datasets.
The authors tackled the problem of assessing neural network architectures by introducing two new metrics, neural efficiency and artificial intelligence quotient (aIQ), to balance performance and size. They demonstrated that a high-aIQ LeNet-5 network was 2.32% less accurate but had 30,912 times fewer parameters and achieved 92.51% accuracy even with 75% randomized labels.
The purpose of this work was to develop of metrics to assess network architectures that balance neural network size and task performance. To this end, the concept of neural efficiency is introduced to measure neural layer utilization, and a second metric called artificial intelligence quotient (aIQ) was created to balance neural network performance and neural network efficiency. To study aIQ and neural efficiency, two simple neural networks were trained on MNIST: a fully connected network (LeNet-300-100) and a convolutional neural network (LeNet-5). The LeNet-5 network with the highest aIQ was 2.32% less accurate but contained 30,912 times fewer parameters than the highest accuracy network. Both batch normalization and dropout layers were found to increase neural efficiency. Finally, high aIQ networks are shown to be memorization and overtraining resistant, capable of learning proper digit classification with an accuracy of 92.51% even when 75% of the class labels are randomized. These results demonstrate the utility of aIQ and neural efficiency as metrics for balancing network performance and size.