Boosting Trust Region Policy Optimization by Normalizing Flows Policy
This work addresses the challenge of efficient exploration in reinforcement learning for high-dimensional tasks, though it appears incremental as it builds on existing trust region methods.
The paper tackled the problem of improving trust region policy optimization by using normalizing flows policies to enhance exploration and avoid local optima, showing significant improvements over baseline architectures on high-dimensional tasks with complex dynamics.
We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps avoid bad local optima. Through extensive comparisons, we show that the normalizing flows policy significantly improves upon baseline architectures especially on high-dimensional tasks with complex dynamics.