Variational Predictive Information Bottleneck
This work provides a theoretical framework for inference in machine learning, but it appears incremental as it builds directly on prior information-theoretic derivations.
The paper tackles the problem of deriving a generalized information-theoretic functional that encompasses modern inference procedures by extending Zellner's work, resulting in a variational lower bound for a predictive information bottleneck objective that suggests novel inference methods.
In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this functional as a variational lower bound of a predictive information bottleneck objective. This generalized functional encompasses most modern inference procedures and suggests novel ones.