On Deep Ensemble Learning from a Function Approximation Perspective
This provides a theoretical foundation for ensemble learning in deep learning, addressing efficiency in model combination, but it is incremental as it builds on existing ensemble and approximation theories.
The paper tackles the problem of function approximation using deep ensemble learning, demonstrating that the framework can universally approximate any function with bounded, sigmoidal, and discriminatory unit models, and shows that deeper ensembles reduce the required number of unit models exponentially, specifically requiring 2d unit models for a single-layer ensemble given input dimension d.
In this paper, we propose to provide a general ensemble learning framework based on deep learning models. Given a group of unit models, the proposed deep ensemble learning framework will effectively combine their learning results via a multilayered ensemble model. In the case when the unit model mathematical mappings are bounded, sigmoidal and discriminatory, we demonstrate that the deep ensemble learning framework can achieve a universal approximation of any functions from the input space to the output space. Meanwhile, to achieve such a performance, the deep ensemble learning framework also impose a strict constraint on the number of involved unit models. According to the theoretic proof provided in this paper, given the input feature space of dimension d, the required unit model number will be 2d, if the ensemble model involves one single layer. Furthermore, as the ensemble component goes deeper, the number of required unit model is proved to be lowered down exponentially.