Majorization Minimization Technique for Optimally Solving Deep Dictionary Learning
This work addresses the suboptimality in deep dictionary learning for researchers in unsupervised learning, offering a more efficient and accurate method, though it is incremental as it builds on existing deep dictionary concepts.
The authors tackled the problem of deep dictionary learning, which previously could only be solved suboptimally with greedy methods, by proposing an optimal solution that simultaneously learns all dictionary layers using the Majorization Minimization approach, resulting in improved performance over greedy learning and outperforming other unsupervised deep learning methods in accuracy and speed on benchmark datasets.
The concept of deep dictionary learning has been recently proposed. Unlike shallow dictionary learning which learns single level of dictionary to represent the data, it uses multiple layers of dictionaries. So far, the problem could only be solved in a greedy fashion; this was achieved by learning a single layer of dictionary in each stage where the coefficients from the previous layer acted as inputs to the subsequent layer (only the first layer used the training samples as inputs). This was not optimal; there was feedback from shallower to deeper layers but not the other way. This work proposes an optimal solution to deep dictionary learning whereby all the layers of dictionaries are solved simultaneously. We employ the Majorization Minimization approach. Experiments have been carried out on benchmark datasets; it shows that optimal learning indeed improves over greedy piecemeal learning. Comparison with other unsupervised deep learning tools (stacked denoising autoencoder, deep belief network, contractive autoencoder and K-sparse autoencoder) show that our method supersedes their performance both in accuracy and speed.