Generalization in quasi-periodic environments
This work addresses the challenge of generalization in machine learning for quasi-periodic environments, presenting an incremental theoretical advancement.
The paper tackles the analysis of stochastic gradient learning in quasi-periodic environments by proposing a dissipative dynamics framework based on the Caldirola-Kanai Hamiltonian, resulting in an asymptotically consistent solution that maps similar patterns to the same decision.
By and large the behavior of stochastic gradient is regarded as a challenging problem, and it is often presented in the framework of statistical machine learning. This paper offers a novel view on the analysis of on-line models of learning that arises when dealing with a generalized version of stochastic gradient that is based on dissipative dynamics. In order to face the complex evolution of these models, a systematic treatment is proposed which is based on energy balance equations that are derived by means of the Caldirola-Kanai (CK) Hamiltonian. According to these equations, learning can be regarded as an ordering process which corresponds with the decrement of the loss function. Finally, the main results established in this paper is that in the case of quasi-periodic environments, where the pattern novelty is progressively limited as time goes by, the system dynamics yields an asymptotically consistent solution in the weight space, that is the solution maps similar patterns to the same decision.