Regulating the information in spikes: a useful bias
This addresses the bias/variance tradeoff in learning for neuroscience and machine learning, but appears incremental as it builds on existing arguments about generic mechanisms.
The paper tackles the problem of balancing model complexity and generalization by proposing a biologically plausible bias that encourages cooperative learning, with rigorous justification provided.
The bias/variance tradeoff is fundamental to learning: increasing a model's complexity can improve its fit on training data, but potentially worsens performance on future samples. Remarkably, however, the human brain effortlessly handles a wide-range of complex pattern recognition tasks. On the basis of these conflicting observations, it has been argued that useful biases in the form of "generic mechanisms for representation" must be hardwired into cortex (Geman et al). This note describes a useful bias that encourages cooperative learning which is both biologically plausible and rigorously justified.