Maximal Sparsity with Deep Networks?
This work addresses the challenge of sparse estimation in domains like 3D scene analysis, offering a novel approach that could enhance accuracy in practical applications, though it is incremental as it builds on existing deep learning and sparse estimation concepts.
The paper tackles the problem of finding maximally sparse representations in signal processing, particularly when dictionaries have coherent columns, by proposing a deep network as a surrogate for traditional sparse estimation algorithms, demonstrating both theoretically and empirically that it can recover minimal ℓ0-norm representations where existing methods fail, with deployment on a photometric stereo estimation problem showing improved outlier removal.
The iterations of many sparse estimation algorithms are comprised of a fixed linear filter cascaded with a thresholding nonlinearity, which collectively resemble a typical neural network layer. Consequently, a lengthy sequence of algorithm iterations can be viewed as a deep network with shared, hand-crafted layer weights. It is therefore quite natural to examine the degree to which a learned network model might act as a viable surrogate for traditional sparse estimation in domains where ample training data is available. While the possibility of a reduced computational budget is readily apparent when a ceiling is imposed on the number of layers, our work primarily focuses on estimation accuracy. In particular, it is well-known that when a signal dictionary has coherent columns, as quantified by a large RIP constant, then most tractable iterative algorithms are unable to find maximally sparse representations. In contrast, we demonstrate both theoretically and empirically the potential for a trained deep network to recover minimal $\ell_0$-norm representations in regimes where existing methods fail. The resulting system is deployed on a practical photometric stereo estimation problem, where the goal is to remove sparse outliers that can disrupt the estimation of surface normals from a 3D scene.