Neither Quick Nor Proper -- Evaluation of QuickProp for Learning Deep Neural Networks
This work addresses the evaluation of an older optimization method for modern deep learning practitioners, showing it is incremental and not beneficial for current applications.
The paper investigated applying QuickProp to train deep neural networks for semantic segmentation, finding that it cannot compete with standard gradient descent methods for this complex task.
Neural networks and especially convolutional neural networks are of great interest in current computer vision research. However, many techniques, extensions, and modifications have been published in the past, which are not yet used by current approaches. In this paper, we study the application of a method called QuickProp for training of deep neural networks. In particular, we apply QuickProp during learning and testing of fully convolutional networks for the task of semantic segmentation. We compare QuickProp empirically with gradient descent, which is the current standard method. Experiments suggest that QuickProp can not compete with standard gradient descent techniques for complex computer vision tasks like semantic segmentation.