Variational Adaptive-Newton Method for Explorative Learning
This work presents a general-purpose method that could improve learning in areas such as active learning and reinforcement learning, though it appears incremental as it unifies existing methods from distinct fields.
The paper tackles the problem of explorative learning in tasks like active learning and reinforcement learning by introducing the Variational Adaptive Newton (VAN) method, a black-box optimization approach that estimates a distribution for exploration with computational efficiency similar to continuous optimization methods, and it performs well across various learning tasks.
We present the Variational Adaptive Newton (VAN) method which is a black-box optimization method especially suitable for explorative-learning tasks such as active learning and reinforcement learning. Similar to Bayesian methods, VAN estimates a distribution that can be used for exploration, but requires computations that are similar to continuous optimization methods. Our theoretical contribution reveals that VAN is a second-order method that unifies existing methods in distinct fields of continuous optimization, variational inference, and evolution strategies. Our experimental results show that VAN performs well on a wide-variety of learning tasks. This work presents a general-purpose explorative-learning method that has the potential to improve learning in areas such as active learning and reinforcement learning.