Overcomplete Dictionary Learning with Jacobi Atom Updates
This work addresses the efficiency and performance of dictionary learning for sparse representations, which is incremental as it modifies the update strategy rather than introducing a new paradigm.
The paper tackled the problem of dictionary learning for sparse representations by proposing a Jacobi version that updates groups of atoms in parallel instead of sequentially. The result showed that parallel algorithms, particularly when all atoms are updated simultaneously, produced better dictionaries than sequential methods, as evidenced by extensive numerical tests on sparse image representation.
Dictionary learning for sparse representations is traditionally approached with sequential atom updates, in which an optimized atom is used immediately for the optimization of the next atoms. We propose instead a Jacobi version, in which groups of atoms are updated independently, in parallel. Extensive numerical evidence for sparse image representation shows that the parallel algorithms, especially when all atoms are updated simultaneously, give better dictionaries than their sequential counterparts.