An Inequality with Applications to Structured Sparsity and Multitask Dictionary Learning
This work provides improved theoretical bounds for structured sparsity and multitask dictionary learning, which is incremental but useful for researchers in machine learning optimization.
The paper derived an inequality from concentration inequalities for Gaussian and Rademacher processes, which was applied to sharpen existing bounds and derive new ones on empirical Rademacher complexities for unit balls in norms related to structured sparsity and multitask dictionary learning, with results depending on the largest eigenvalue of the data covariance matrix.
From concentration inequalities for the suprema of Gaussian or Rademacher processes an inequality is derived. It is applied to sharpen existing and to derive novel bounds on the empirical Rademacher complexities of unit balls in various norms appearing in the context of structured sparsity and multitask dictionary learning or matrix factorization. A key role is played by the largest eigenvalue of the data covariance matrix.