Distilling Symbolic Priors for Concept Learning into Neural Networks
This addresses the challenge of improving few-shot learning in AI by transferring symbolic priors into neural networks, representing an incremental advance in hybrid AI methods.
The paper tackled the problem of enabling neural networks to learn new concepts from few examples by instilling human-like inductive biases, achieving models that closely align with human performance in learning logical concepts.
Humans can learn new concepts from a small number of examples by drawing on their inductive biases. These inductive biases have previously been captured by using Bayesian models defined over symbolic hypothesis spaces. Is it possible to create a neural network that displays the same inductive biases? We show that inductive biases that enable rapid concept learning can be instantiated in artificial neural networks by distilling a prior distribution from a symbolic Bayesian model via meta-learning, an approach for extracting the common structure from a set of tasks. By generating the set of tasks used in meta-learning from the prior distribution of a Bayesian model, we are able to transfer that prior into a neural network. We use this approach to create a neural network with an inductive bias towards concepts expressed as short logical formulas. Analyzing results from previous behavioral experiments in which people learned logical concepts from a few examples, we find that our meta-trained models are highly aligned with human performance.