Boosting Dictionary Learning with Error Codes
This work addresses a bottleneck in dictionary learning for high-dimensional data, offering incremental improvements to convergence and stability in sparse coding applications.
The paper tackles the problem of improper initial dictionaries in sparse representation dictionary learning, which can lead to suboptimal convergence, by introducing a feedback process that codes intermediate error and adjusts sparse codes, resulting in vastly improved convergence rates and better converged states, especially with random initializations and lenient sparsity constraints.
In conventional sparse representations based dictionary learning algorithms, initial dictionaries are generally assumed to be proper representatives of the system at hand. However, this may not be the case, especially in some systems restricted to random initializations. Therefore, a supposedly optimal state-update based on such an improper model might lead to undesired effects that will be conveyed to successive iterations. In this paper, we propose a dictionary learning method which includes a general feedback process that codes the intermediate error left over from a less intensive initial learning attempt, and then adjusts sparse codes accordingly. Experimental observations show that such an additional step vastly improves rates of convergence in high-dimensional cases, also results in better converged states in the case of random initializations. Improvements also scale up with more lenient sparsity constraints.