The Theory and Algorithm of Ergodic Inference
This foundational work aims to address a core problem in machine learning for researchers and practitioners by introducing a novel theoretical framework, though it is incremental as it focuses on theory without practical algorithm implementation.
The paper tackles the challenge of improving computational scalability and statistical efficiency in approximate inference by proposing a new theoretical framework called ergodic inference, based on ergodic transformations, to overcome limitations in existing variational inference and Markov chain Monte Carlo methods.
Approximate inference algorithm is one of the fundamental research fields in machine learning. The two dominant theoretical inference frameworks in machine learning are variational inference (VI) and Markov chain Monte Carlo (MCMC). However, because of the fundamental limitation in the theory, it is very challenging to improve existing VI and MCMC methods on both the computational scalability and statistical efficiency. To overcome this obstacle, we propose a new theoretical inference framework called ergodic Inference based on the fundamental property of ergodic transformations. The key contribution of this work is to establish the theoretical foundation of ergodic inference for the development of practical algorithms in future work.