Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
This addresses the challenge of non-stationarity in machine learning for applications like continual learning and reinforcement learning, though it appears incremental as it builds on existing adaptation techniques.
The paper tackles the problem of neural network training under non-stationary data distributions, such as in distributional shifts and continual learning, by introducing a method that automatically adapts via an Ornstein-Uhlenbeck process with adaptive drift, resulting in effective performance in supervised and reinforcement learning settings.
Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.