Online Learning with Gated Linear Networks
This provides a theoretically grounded approach for online learning in probabilistic settings, though it appears incremental as it builds on existing linear network concepts with a new conditioning twist.
The paper tackles the problem of online learning under logarithmic loss by introducing a family of probabilistic architectures that use data conditioning instead of non-linear transfer functions for representational power. It proves a learnable capacity theorem showing these models can learn any bounded Borel-measurable function on compact Euclidean subsets, with guaranteed convergence for both the model and learning procedure.
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditioning. We state under general conditions a learnable capacity theorem that shows this approach can in principle learn any bounded Borel-measurable function on a compact subset of euclidean space; the result is stronger than many universality results for connectionist architectures because we provide both the model and the learning procedure for which convergence is guaranteed.