Learning Successor Features with Distributed Hebbian Temporal Memory
This addresses the problem of decision-making under uncertainty in dynamic environments for AI systems, representing an incremental improvement with a novel method.
The paper tackles online sequence learning in non-stationary, partially observable environments by proposing Distributed Hebbian Temporal Memory (DHTM), which outperforms LSTM, RWKV, and CSCG on non-stationary datasets.
This paper presents a novel approach to address the challenge of online sequence learning for decision making under uncertainty in non-stationary, partially observable environments. The proposed algorithm, Distributed Hebbian Temporal Memory (DHTM), is based on the factor graph formalism and a multi-component neuron model. DHTM aims to capture sequential data relationships and make cumulative predictions about future observations, forming Successor Features (SFs). Inspired by neurophysiological models of the neocortex, the algorithm uses distributed representations, sparse transition matrices, and local Hebbian-like learning rules to overcome the instability and slow learning of traditional temporal memory algorithms such as RNN and HMM. Experimental results show that DHTM outperforms LSTM, RWKV and a biologically inspired HMM-like algorithm, CSCG, on non-stationary data sets. Our results suggest that DHTM is a promising approach to address the challenges of online sequence learning and planning in dynamic environments.