Universally Consistent Online Learning with Arbitrarily Dependent Responses
This work addresses the theoretical foundations of online learning for researchers, showing that ergodicity is unnecessary for universal consistency, which is an incremental advance in relaxing assumptions.
The paper tackles the problem of universally consistent online learning under arbitrarily dependent responses, presenting an online learning rule that achieves universal consistency with conditions only on the X process, generalizing past results that required stationarity and ergodicity for the joint process.
This work provides an online learning rule that is universally consistent under processes on (X,Y) pairs, under conditions only on the X process. As a special case, the conditions admit all processes on (X,Y) such that the process on X is stationary. This generalizes past results which required stationarity for the joint process on (X,Y), and additionally required this process to be ergodic. In particular, this means that ergodicity is superfluous for the purpose of universally consistent online learning.