A Forward Backward Greedy approach for Sparse Multiscale Learning
This work addresses the need for efficient multiscale learning in data analysis, though it appears incremental by building on existing RKHS and greedy methods.
The authors tackled the problem of constructing sparse multiscale representations in a feature-driven RKHS by proposing a forward-backward greedy algorithm, which achieved efficient predictions and data reduction as demonstrated on simulation and real datasets.
Multiscale Models are known to be successful in uncovering and analyzing the structures in data at different resolutions. In the current work we propose a feature driven Reproducing Kernel Hilbert space (RKHS), for which the associated kernel has a weighted multiscale structure. For generating approximations in this space, we provide a practical forward-backward algorithm that is shown to greedily construct a set of basis functions having a multiscale structure, while also creating sparse representations from the given data set, making representations and predictions very efficient. We provide a detailed analysis of the algorithm including recommendations for selecting algorithmic hyper-parameters and estimating probabilistic rates of convergence at individual scales. Then we extend this analysis to multiscale setting, studying the effects of finite scale truncation and quality of solution in the inherent RKHS. In the last section, we analyze the performance of the approach on a variety of simulation and real data sets, thereby justifying the efficiency claims in terms of model quality and data reduction.