The Use of Mutual Coherence to Prove $\ell^1/\ell^0$-Equivalence in Classification Problems
This addresses theoretical limitations in classification methods for machine learning researchers, but it is incremental as it builds on existing sparse representation frameworks.
The paper tackles the problem of verifying when ℓ¹-minimization retrieves the sparsest solution in sparse representation-based classification, showing that deterministic equivalence conflicts with high coherence but approximate equivalence works with well-separated classes.
We consider the decomposition of a signal over an overcomplete set of vectors. Minimization of the $\ell^1$-norm of the coefficient vector can often retrieve the sparsest solution (so-called "$\ell^1/\ell^0$-equivalence"), a generally NP-hard task, and this fact has powered the field of compressed sensing. Wright et al.'s sparse representation-based classification (SRC) applies this relationship to machine learning, wherein the signal to be decomposed represents the test sample and columns of the dictionary are training samples. We investigate the relationships between $\ell^1$-minimization, sparsity, and classification accuracy in SRC. After proving that the tractable, deterministic approach to verifying $\ell^1/\ell^0$-equivalence fundamentally conflicts with the high coherence between same-class training samples, we demonstrate that $\ell^1$-minimization can still recover the sparsest solution when the classes are well-separated. Further, using a nonlinear transform so that sparse recovery conditions may be satisfied, we demonstrate that approximate (not strict) equivalence is key to the success of SRC.