Simple2Complex: Global Optimization by Gradient Descent
This addresses the issue of local minima in deep learning training, potentially benefiting researchers and practitioners in neural network optimization, though it appears incremental as it builds on existing training methods.
The paper tackles the problem of deep neural networks getting stuck in local minima during training by proposing a method called simple2complex, which gradually adds layers to shallow networks as training progresses, resulting in improved global optimization capabilities as verified on CIFAR-10.
A method named simple2complex for modeling and training deep neural networks is proposed. Simple2complex train deep neural networks by smoothly adding more and more layers to the shallow networks, as the learning procedure going on, the network is just like growing. Compared with learning by end2end, simple2complex is with less possibility trapping into local minimal, namely, owning ability for global optimization. Cifar10 is used for verifying the superiority of simple2complex.