Linear discriminant initialization for feed-forward neural networks
This addresses the training efficiency and performance issue for neural network practitioners, though it appears incremental as it builds on existing initialization methods.
The paper tackles the problem of slow training and suboptimal accuracy in feed-forward neural networks by initializing the first layer's weights using linear discriminants that best distinguish individual classes, resulting in fewer training steps and higher asymptotic accuracy on training data.
Informed by the basic geometry underlying feed forward neural networks, we initialize the weights of the first layer of a neural network using the linear discriminants which best distinguish individual classes. Networks initialized in this way take fewer training steps to reach the same level of training, and asymptotically have higher accuracy on training data.