Effective Sparsity: A Unified Framework via Normalized Entropy and the Effective Number of Nonzeros
This addresses a limitation in sparsity-based reconstruction for inverse problems, offering a more stable measure, though it is incremental as it builds on existing regularization frameworks.
The paper tackles the problem of overestimating signal complexity in sparsity-promoting methods by introducing effective sparsity via normalized entropy-based regularizers, showing that this approach outperforms traditional methods in robustness and accuracy in noisy linear inverse problems.
Classical sparsity promoting methods rely on the l0 norm, which treats all nonzero components as equally significant. In practical inverse problems, however, solutions often exhibit many small amplitude components that have little effect on reconstruction but lead to an overestimation of signal complexity. We address this limitation by shifting the paradigm from discrete cardinality to effective sparsity. Our approach introduces the effective number of nonzeros (ENZ), a unified class of normalized entropy-based regularizers, including Shannon and Renyi forms, that quantifies the concentration of significant coefficients. We show that, unlike the classical l0 norm, the ENZ provides a stable and continuous measure of effective sparsity that is insensitive to negligible perturbations. For noisy linear inverse problems, we establish theoretical guarantees under the Restricted Isometry Property (RIP), proving that ENZ based recovery is unique and stable. We also derive a decomposition showing that the ENZ equals the support cardinality times a distributional efficiency term, thereby linking entropy with l0 regularization. Numerical experiments show that this effective sparsity framework outperforms traditional cardinality based methods in robustness and accuracy.