Proximal Bellman mappings for reinforcement learning and their application to robust adaptive filtering
This work addresses robust adaptive filtering for signal processing applications, presenting an incremental improvement by applying a novel RL method to a specific domain problem.
The paper tackles the problem of selecting the optimal p-norm loss exponent online to combat outliers in linear adaptive filtering without prior data or outlier statistics, achieving superior performance over existing non-RL and kernel-based RL schemes in numerical tests.
This paper aims at the algorithmic/theoretical core of reinforcement learning (RL) by introducing the novel class of proximal Bellman mappings. These mappings are defined in reproducing kernel Hilbert spaces (RKHSs), to benefit from the rich approximation properties and inner product of RKHSs, they are shown to belong to the powerful Hilbertian family of (firmly) nonexpansive mappings, regardless of the values of their discount factors, and possess ample degrees of design freedom to even reproduce attributes of the classical Bellman mappings and to pave the way for novel RL designs. An approximate policy-iteration scheme is built on the proposed class of mappings to solve the problem of selecting online, at every time instance, the "optimal" exponent $p$ in a $p$-norm loss to combat outliers in linear adaptive filtering, without training data and any knowledge on the statistical properties of the outliers. Numerical tests on synthetic data showcase the superior performance of the proposed framework over several non-RL and kernel-based RL schemes.