Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization
This addresses the challenge of efficiently training complex models for machine learning practitioners, though it appears incremental as it builds on existing optimization methods.
The paper tackled the problem of training nonconvex models like neural networks by proposing a successive functional gradient optimization framework, and it resulted in better performance compared to standard training techniques.
This paper presents a framework of successive functional gradient optimization for training nonconvex models such as neural networks, where training is driven by mirror descent in a function space. We provide a theoretical analysis and empirical study of the training method derived from this framework. It is shown that the method leads to better performance than that of standard training techniques.