WEEP: A Differentiable Nonconvex Sparse Regularizer via Weakly-Convex Envelope
This addresses a fundamental dilemma in signal processing for efficient signal recovery and feature extraction, offering a novel solution that is incremental in improving computational tractability.
The paper tackles the conflict between powerful non-differentiable sparse regularizers and gradient-based optimizers by introducing WEEP, a differentiable sparse regularizer derived from the weakly-convex envelope framework, which achieves superior performance in signal and image denoising tasks compared to L1-norm and other non-convex regularizers.
Sparse regularization is fundamental in signal processing for efficient signal recovery and feature extraction. However, it faces a fundamental dilemma: the most powerful sparsity-inducing penalties are often non-differentiable, conflicting with gradient-based optimizers that dominate the field. We introduce WEEP (Weakly-convex Envelope of Piecewise Penalty), a novel, fully differentiable sparse regularizer derived from the weakly-convex envelope framework. WEEP provides strong, unbiased sparsity while maintaining full differentiability and L-smoothness, making it natively compatible with any gradient-based optimizer. This resolves the conflict between statistical performance and computational tractability. We demonstrate superior performance compared to the L1-norm and other established non-convex sparse regularizers on challenging signal and image denoising tasks.