Assessing Simplification Levels in Neural Networks: The Impact of Hyperparameter Configurations on Complexity and Sensitivity
This study addresses the theoretical understanding of neural network complexity and sensitivity for researchers, but it is incremental as it applies existing methods to analyze hyperparameter effects without introducing new techniques.
This paper tackled the problem of understanding how hyperparameter configurations affect the simplification properties of neural networks, specifically measuring Lempel Ziv complexity and sensitivity, and found that adjustments to activation functions, hidden layers, and learning rate impact these metrics in experiments on the MNIST dataset.
This paper presents an experimental study focused on understanding the simplification properties of neural networks under different hyperparameter configurations, specifically investigating the effects on Lempel Ziv complexity and sensitivity. By adjusting key hyperparameters such as activation functions, hidden layers, and learning rate, this study evaluates how these parameters impact the complexity of network outputs and their robustness to input perturbations. The experiments conducted using the MNIST dataset aim to provide insights into the relationships between hyperparameters, complexity, and sensitivity, contributing to a deeper theoretical understanding of these concepts in neural networks.