On the Role of Time in Learning
This work addresses a foundational issue in machine learning by rethinking how time is incorporated, potentially impacting all temporal learning tasks.
The paper argues that traditional risk minimization methods for temporal learning may overlook deeper interpretations of time, and proposes using the principle of Least Cognitive Action to derive differential equations that align learning with natural laws.
By and large the process of learning concepts that are embedded in time is regarded as quite a mature research topic. Hidden Markov models, recurrent neural networks are, amongst others, successful approaches to learning from temporal data. In this paper, we claim that the dominant approach minimizing appropriate risk functions defined over time by classic stochastic gradient might miss the deep interpretation of time given in other fields like physics. We show that a recent reformulation of learning according to the principle of Least Cognitive Action is better suited whenever time is involved in learning. The principle gives rise to a learning process that is driven by differential equations, that can somehow descrive the process within the same framework as other laws of nature.