On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning
This work addresses computational efficiency in reinforcement learning for researchers, though it appears incremental as it builds on existing multiscale methods.
The paper tackles the problem of value function approximation in reinforcement learning by proposing an adaptive multiscale framework using multiresolution analysis and tree approximation, achieving a convergence rate independent of basis function regularity.
In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions.