Boltzmann machines for time-series
This addresses computational inefficiency in online learning for time-series models, but it is incremental as it reviews and extends existing methods.
The paper reviews Boltzmann machines for time-series, highlighting that backpropagation through time (BPTT) has high computational complexity in online learning, and introduces dynamic Boltzmann machines (DyBMs) with a local-in-time learning rule related to spike-timing dependent plasticity (STDP).
We review Boltzmann machines extended for time-series. These models often have recurrent structure, and back propagration through time (BPTT) is used to learn their parameters. The per-step computational complexity of BPTT in online learning, however, grows linearly with respect to the length of preceding time-series (i.e., learning rule is not local in time), which limits the applicability of BPTT in online learning. We then review dynamic Boltzmann machines (DyBMs), whose learning rule is local in time. DyBM's learning rule relates to spike-timing dependent plasticity (STDP), which has been postulated and experimentally confirmed for biological neural networks.