SelfieBoost: A Boosting Algorithm for Deep Learning
This work addresses the need for more efficient boosting methods in deep learning, but it appears incremental as it adapts boosting concepts to single networks rather than introducing a new paradigm.
The authors tackled the problem of improving deep learning accuracy by introducing SelfieBoost, a boosting algorithm that enhances a single network's performance rather than constructing ensembles, and they proved a log(1/ε) convergence rate under a practical assumption.
We describe and analyze a new boosting algorithm for deep learning called SelfieBoost. Unlike other boosting algorithms, like AdaBoost, which construct ensembles of classifiers, SelfieBoost boosts the accuracy of a single network. We prove a $\log(1/ε)$ convergence rate for SelfieBoost under some "SGD success" assumption which seems to hold in practice.