Supervised structure learning
This work addresses the problem of learning generative models with structured latent states for researchers in machine learning, though it appears incremental as it builds on existing Bayesian and free energy frameworks.
The paper tackles structure learning of discrete generative models by using Bayesian model selection with priors based on expected free energy, which reduces to constrained mutual information. It demonstrates the approach on MNIST image classification and more complex tasks like visual disentanglement and the Tower of Hanoi problem, recovering latent factorial structures and dynamics.
This paper concerns structure learning or discovery of discrete generative models. It focuses on Bayesian model selection and the assimilation of training data or content, with a special emphasis on the order in which data are ingested. A key move - in the ensuing schemes - is to place priors on the selection of models, based upon expected free energy. In this setting, expected free energy reduces to a constrained mutual information, where the constraints inherit from priors over outcomes (i.e., preferred outcomes). The resulting scheme is first used to perform image classification on the MNIST dataset to illustrate the basic idea, and then tested on a more challenging problem of discovering models with dynamics, using a simple sprite-based visual disentanglement paradigm and the Tower of Hanoi (cf., blocks world) problem. In these examples, generative models are constructed autodidactically to recover (i.e., disentangle) the factorial structure of latent states - and their characteristic paths or dynamics.