Information in Infinite Ensembles of Infinitely-Wide Neural Networks

This work addresses the problem of understanding generalization in neural networks for researchers, but it is incremental as it builds on existing infinite-width theories.

The authors studied the generalization properties of infinite ensembles of infinitely-wide neural networks, finding that this model family allows for tractable calculations of information-theoretic quantities, with analytical and empirical investigations conducted to identify signals correlating with generalization.