Advantages of Neural Population Coding for Deep Learning
This work addresses the problem of improving neural network robustness and accuracy for ambiguous predictions, though it is incremental as it builds on known biological coding principles.
The paper investigates using neural population coding in the output layer of neural networks, showing it improves robustness to input noise in linear networks and enhances accuracy for ambiguous outputs like 3D object orientation on the T-LESS dataset.
Scalar variables, e.g., the orientation of a shape in an image, are commonly predicted using a single output neuron in a neural network. In contrast, the mammalian cortex represents variables with a population of neurons. In this population code, each neuron is most active at its preferred value and shows partial activity for other values. Here, we investigate the benefit of using a population code for the output layer of a neural network. We compare population codes against single-neuron outputs and one-hot vectors. First, we show theoretically and in experiments with synthetic data that population codes improve robustness to input noise in networks of stacked linear layers. Second, we demonstrate the benefit of using population codes to encode ambiguous outputs, such as the pose of symmetric objects. Using the T-LESS dataset of feature-less real-world objects, we show that population codes improve the accuracy of predicting 3D object orientation from image input.