Learning to learn generative programs with Memoised Wake-Sleep
This work addresses the problem of learning explainable generative models for researchers in neuro-symbolic AI, representing an incremental improvement over existing Wake-Sleep methods.
The paper tackles the challenge of performing program induction as an inner-loop in learning neuro-symbolic generative models by proposing the Memoised Wake-Sleep (MWS) algorithm, which stores and reuses the best programs discovered during training, and applies it to achieve accurate, explainable models in domains like stroke-based character modelling, cellular automata, and few-shot learning on real-world string concepts.
We study a class of neuro-symbolic generative models in which neural networks are used both for inference and as priors over symbolic, data-generating programs. As generative models, these programs capture compositional structures in a naturally explainable form. To tackle the challenge of performing program induction as an 'inner-loop' to learning, we propose the Memoised Wake-Sleep (MWS) algorithm, which extends Wake Sleep by explicitly storing and reusing the best programs discovered by the inference network throughout training. We use MWS to learn accurate, explainable models in three challenging domains: stroke-based character modelling, cellular automata, and few-shot learning in a novel dataset of real-world string concepts.