Star algorithm for NN ensembling
This work addresses the need for more efficient and theoretically grounded ensembling methods in machine learning, though it appears incremental as it builds on existing star algorithms.
The authors tackled the problem of neural network ensembling by proposing a new algorithm based on Audibert's empirical star algorithm, achieving an optimal theoretical minimax bound on excess squared risk and demonstrating empirical performance on regression and classification tasks compared to popular methods.
Neural network ensembling is a common and robust way to increase model efficiency. In this paper, we propose a new neural network ensemble algorithm based on Audibert's empirical star algorithm. We provide optimal theoretical minimax bound on the excess squared risk. Additionally, we empirically study this algorithm on regression and classification tasks and compare it to most popular ensembling methods.