Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics
This work addresses uncertainty-aware dynamics learning for control or robotics, presenting a novel hybrid approach that combines kernel methods and neural networks.
The paper tackled the problem of learning uncertain system dynamics by modeling them as unknown smooth functions in reproducing kernel Hilbert spaces, using random features with uncertain parameters equivalent to a shallow Bayesian neural network, and showed that finding worst-case dynamics via Pontryagin's minimum principle is equivalent to the Frank-Wolfe algorithm on a deep network, with numerical experiments demonstrating the method's capacity.
Random features is a powerful universal function approximator that inherits the theoretical rigor of kernel methods and can scale up to modern learning tasks. This paper views uncertain system models as unknown or uncertain smooth functions in universal reproducing kernel Hilbert spaces. By directly approximating the one-step dynamics function using random features with uncertain parameters, which are equivalent to a shallow Bayesian neural network, we then view the whole dynamical system as a multi-layer neural network. Exploiting the structure of Hamiltonian dynamics, we show that finding worst-case dynamics realizations using Pontryagin's minimum principle is equivalent to performing the Frank-Wolfe algorithm on the deep net. Various numerical experiments on dynamics learning showcase the capacity of our modeling methodology.