Online Multilinear Dictionary Learning
This work addresses computational efficiency for real-time tensor processing, but it appears incremental as it builds on existing dictionary learning methods with specific optimizations.
The paper tackles the problem of online tensor dictionary learning by proposing a method that reduces computational complexity from O(L^N) to O(NL^2) using tensor contraction and ensures convergence with stochastic gradient techniques. Experiments on synthetic signals confirm impressive performance, though no specific numbers are provided.
A method for online tensor dictionary learning is proposed. With the assumption of separable dictionaries, tensor contraction is used to diminish a $N$-way model of $\mathcal{O}\left(L^N\right)$ into a simple matrix equation of $\mathcal{O}\left(NL^2\right)$ with a real-time capability. To avoid numerical instability due to inversion of sparse matrix, a class of stochastic gradient with memory is formulated via a least-square solution to guarantee convergence and robustness. Both gradient descent with exact line search and Newton's method are discussed and realized. Extensions onto how to deal with bad initialization and outliers are also explained in detail. Experiments on two synthetic signals confirms an impressive performance of our proposed method.