Learning Efficient Structured Sparse Models
This work addresses the need for real-time and large-scale applications in machine learning by providing a faster framework for structured sparse modeling, though it is incremental as it builds on existing ideas of learnable fast regressors.
The paper tackles the problem of efficiently computing structured sparse codes by introducing a novel block-coordinate proximal splitting method and a feed-forward architecture that approximates exact codes with much lower complexity. It demonstrates several orders of magnitude speedup compared to state-of-the-art methods with minimal performance degradation.
We present a comprehensive framework for structured sparse coding and modeling extending the recent ideas of using learnable fast regressors to approximate exact sparse codes. For this purpose, we develop a novel block-coordinate proximal splitting method for the iterative solution of hierarchical sparse coding problems, and show an efficient feed forward architecture derived from its iteration. This architecture faithfully approximates the exact structured sparse codes with a fraction of the complexity of the standard optimization methods. We also show that by using different training objective functions, learnable sparse encoders are no longer restricted to be mere approximants of the exact sparse code for a pre-given dictionary, as in earlier formulations, but can be rather used as full-featured sparse encoders or even modelers. A simple implementation shows several orders of magnitude speedup compared to the state-of-the-art at minimal performance degradation, making the proposed framework suitable for real time and large-scale applications.