A Theory of Human-Like Few-Shot Learning
This work addresses the challenge of enabling machines to learn from few samples like humans, which is incremental as it builds on existing theories and models.
The paper tackles the problem of bridging human-like few-shot learning with machine learning by deriving a theory from von-Neuman-Landauer's principle, showing that deep generative models like VAEs approximate this theory and outperform baseline models in tasks such as image recognition and low-resource language processing.
We aim to bridge the gap between our common-sense few-sample human learning and large-data machine learning. We derive a theory of human-like few-shot learning from von-Neuman-Landauer's principle. modelling human learning is difficult as how people learn varies from one to another. Under commonly accepted definitions, we prove that all human or animal few-shot learning, and major models including Free Energy Principle and Bayesian Program Learning that model such learning, approximate our theory, under Church-Turing thesis. We find that deep generative model like variational autoencoder (VAE) can be used to approximate our theory and perform significantly better than baseline models including deep neural networks, for image recognition, low resource language processing, and character recognition.