Improving Sparsity in Kernel Adaptive Filters Using a Unit-Norm Dictionary
This work addresses computational efficiency issues in kernel adaptive filters for time-series modeling, but it is incremental as it builds on existing methods with a specific modification.
The paper tackled the problem of kernel adaptive filters struggling with accurate predictions and computational complexity for trivial monotonic signals by proposing a unit-norm dictionary approach, which reduced dictionary size by up to 30% while maintaining competitive normalized mean square error on real-world datasets.
Kernel adaptive filters, a class of adaptive nonlinear time-series models, are known by their ability to learn expressive autoregressive patterns from sequential data. However, for trivial monotonic signals, they struggle to perform accurate predictions and at the same time keep computational complexity within desired boundaries. This is because new observations are incorporated to the dictionary when they are far from what the algorithm has seen in the past. We propose a novel approach to kernel adaptive filtering that compares new observations against dictionary samples in terms of their unit-norm (normalised) versions, meaning that new observations that look like previous samples but have a different magnitude are not added to the dictionary. We achieve this by proposing the unit-norm Gaussian kernel and define a sparsification criterion for this novel kernel. This new methodology is validated on two real-world datasets against standard KAF in terms of the normalised mean square error and the dictionary size.