Randomized co-training: from cortical neurons to machine learning and back again
This addresses a fundamental challenge in neuroscience and machine learning by bridging cortical learning principles with scalable algorithms, though it appears incremental in applying existing co-training ideas to a biological context.
The paper tackles the problem of how neurons can leverage unlabeled data to accelerate learning from sparse labels, proposing randomized co-training as a biologically plausible meta-algorithm. It presents Correlated Nystrom Views (XNV), which achieves state-of-the-art performance in semi-supervised learning, and outlines ongoing work on a neuronal implementation.
Despite its size and complexity, the human cortex exhibits striking anatomical regularities, suggesting there may simple meta-algorithms underlying cortical learning and computation. We expect such meta-algorithms to be of interest since they need to operate quickly, scalably and effectively with little-to-no specialized assumptions. This note focuses on a specific question: How can neurons use vast quantities of unlabeled data to speed up learning from the comparatively rare labels provided by reward systems? As a partial answer, we propose randomized co-training as a biologically plausible meta-algorithm satisfying the above requirements. As evidence, we describe a biologically-inspired algorithm, Correlated Nystrom Views (XNV) that achieves state-of-the-art performance in semi-supervised learning, and sketch work in progress on a neuronal implementation.