Confidence Dimension for Deep Learning based on Hoeffding Inequality and Relative Evaluation
This addresses the open problem of quantifying generalization in complex DNNs, including binary neural networks, but is incremental as it builds on existing theoretical concepts.
The paper tackles the challenge of measuring generalization ability in deep neural networks by proposing a confidence dimension (CD) framework that uses multiple factors and theoretical bounds based on VC-dimension and Hoeffding's inequality, achieving consistent ranking for full-precision and binary neural networks on image classification and object detection tasks.
Research on the generalization ability of deep neural networks (DNNs) has recently attracted a great deal of attention. However, due to their complex architectures and large numbers of parameters, measuring the generalization ability of specific DNN models remains an open challenge. In this paper, we propose to use multiple factors to measure and rank the relative generalization of DNNs based on a new concept of confidence dimension (CD). Furthermore, we provide a feasible framework in our CD to theoretically calculate the upper bound of generalization based on the conventional Vapnik-Chervonenk dimension (VC-dimension) and Hoeffding's inequality. Experimental results on image classification and object detection demonstrate that our CD can reflect the relative generalization ability for different DNNs. In addition to full-precision DNNs, we also analyze the generalization ability of binary neural networks (BNNs), whose generalization ability remains an unsolved problem. Our CD yields a consistent and reliable measure and ranking for both full-precision DNNs and BNNs on all the tasks.