Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flow
This provides a theoretical framework for sparse approximation in machine learning, though it appears incremental as it builds on existing flow methods.
The paper tackles the problem of finding sparse representations of Barron functions by using the inverse scale space flow to minimize L^2 loss, achieving convergence rates of O(1/t) in ideal settings and up to constants with noise or bias.
This paper presents a method for finding a sparse representation of Barron functions. Specifically, given an $L^2$ function $f$, the inverse scale space flow is used to find a sparse measure $μ$ minimising the $L^2$ loss between the Barron function associated to the measure $μ$ and the function $f$. The convergence properties of this method are analysed in an ideal setting and in the cases of measurement noise and sampling bias. In an ideal setting the objective decreases strictly monotone in time to a minimizer with $\mathcal{O}(1/t)$, and in the case of measurement noise or sampling bias the optimum is achieved up to a multiplicative or additive constant. This convergence is preserved on discretization of the parameter space, and the minimizers on increasingly fine discretizations converge to the optimum on the full parameter space.