Rethinking supervised learning: insights from biological learning and from calling it by its name
This paper addresses a conceptual problem for the machine learning community regarding the terminology and fundamental understanding of learning paradigms, arguing for a more grounded perspective.
This paper argues that the term "supervised learning" is often misconstrued and that alternatives like unsupervised or self-supervised learning are merely brand names rather than distinct theoretical categories. It suggests that progress in AI will be better served by acknowledging the necessity of supervision and inductive biases, drawing insights from biological learning.
The renaissance of artificial neural networks was catalysed by the success of classification models, tagged by the community with the broader term supervised learning. The extraordinary results gave rise to a hype loaded with ambitious promises and overstatements. Soon the community realised that the success owed much to the availability of thousands of labelled examples and supervised learning went, for many, from glory to shame: Some criticised deep learning as a whole and others proclaimed that the way forward had to be alternatives to supervised learning: predictive, unsupervised, semi-supervised and, more recently, self-supervised learning. However, all these seem brand names, rather than actual categories of a theoretically grounded taxonomy. Moreover, the call to banish supervised learning was motivated by the questionable claim that humans learn with little or no supervision and are capable of robust out-of-distribution generalisation. Here, we review insights about learning and supervision in nature, revisit the notion that learning and generalisation are not possible without supervision or inductive biases and argue that we will make better progress if we just call it by its name.