Spatio-Temporal Activation Function To Map Complex Dynamical Systems
This work addresses a problem for researchers and practitioners in machine learning and dynamical systems modeling, but it appears incremental as it builds on existing activation function concepts.
The authors tackled the challenge of modeling complex dynamical systems by proposing a two-dimensional activation function with a temporal term, enabling the capture of time series dynamics without recurrent neural networks, though no concrete performance numbers are provided.
Most of the real world is governed by complex and chaotic dynamical systems. All of these dynamical systems pose a challenge in modelling them using neural networks. Currently, reservoir computing, which is a subset of recurrent neural networks, is actively used to simulate complex dynamical systems. In this work, a two dimensional activation function is proposed which includes an additional temporal term to impart dynamic behaviour on its output. The inclusion of a temporal term alters the fundamental nature of an activation function, it provides capability to capture the complex dynamics of time series data without relying on recurrent neural networks.