Cooperation Is All You Need
This work addresses the challenge of improving learning efficiency in reinforcement learning, offering a potential advancement over existing methods like Transformers, though it appears incremental in its specific application.
The paper tackles the problem of slow learning in reinforcement learning by introducing Cooperator, a novel neural network architecture inspired by pyramidal neurons, which learns significantly faster than Transformer-based algorithms while using the same number of parameters.
Going beyond 'dendritic democracy', we introduce a 'democracy of local processors', termed Cooperator. Here we compare their capabilities when used in permutation invariant neural networks for reinforcement learning (RL), with machine learning algorithms based on Transformers, such as ChatGPT. Transformers are based on the long standing conception of integrate-and-fire 'point' neurons, whereas Cooperator is inspired by recent neurobiological breakthroughs suggesting that the cellular foundations of mental life depend on context-sensitive pyramidal neurons in the neocortex which have two functionally distinct points. Weshow that when used for RL, an algorithm based on Cooperator learns far quicker than that based on Transformer, even while having the same number of parameters.