Hierarchical regularization networks for sparsification based learning on noisy datasets
This work addresses data reduction and modeling for noisy datasets, but it appears incremental as it builds on existing sparsification and regularization techniques.
The paper tackles the problem of learning from large noisy datasets by proposing a hierarchical regularization network that generates sparse representations and models, achieving efficient data reduction and preserving generalizability on synthetic and real datasets.
We propose a hierarchical learning strategy aimed at generating sparse representations and associated models for large noisy datasets. The hierarchy follows from approximation spaces identified at successively finer scales. For promoting model generalization at each scale, we also introduce a novel, projection based penalty operator across multiple dimension, using permutation operators for incorporating proximity and ordering information. The paper presents a detailed analysis of approximation properties in the reconstruction Reproducing Kernel Hilbert Spaces (RKHS) with emphasis on optimality and consistency of predictions and behavior of error functionals associated with the produced sparse representations. Results show the performance of the approach as a data reduction and modeling strategy on both synthetic (univariate and multivariate) and real datasets (time series). The sparse model for the test datasets, generated by the presented approach, is also shown to efficiently reconstruct the underlying process and preserve generalizability.