First and Second Order Methods for Online Convolutional Dictionary Learning
This work addresses memory limitations for researchers and practitioners using convolutional sparse representations, though it appears incremental as it builds on prior work.
The paper tackles the problem of high memory usage in convolutional dictionary learning by developing online algorithms that scale better with training data size, proposing a new algorithm with improved performance and support for incomplete data, and providing theoretical analysis.
Convolutional sparse representations are a form of sparse representation with a structured, translation invariant dictionary. Most convolutional dictionary learning algorithms to date operate in batch mode, requiring simultaneous access to all training images during the learning process, which results in very high memory usage and severely limits the training data that can be used. Very recently, however, a number of authors have considered the design of online convolutional dictionary learning algorithms that offer far better scaling of memory and computational cost with training set size than batch methods. This paper extends our prior work, improving a number of aspects of our previous algorithm; proposing an entirely new one, with better performance, and that supports the inclusion of a spatial mask for learning from incomplete data; and providing a rigorous theoretical analysis of these methods.