From generative AI to the brain: five takeaways
It proposes a cross-disciplinary approach for neuroscience to learn from ML, but is incremental as it builds on existing generative AI concepts.
The paper argues that generative AI's success stems from clear generative principles, not obscure algorithms, and suggests neuroscience should investigate which of these principles apply to the brain, using examples like attention and neural scaling laws.
The big strides seen in generative AI are not based on somewhat obscure algorithms, but due to clearly defined generative principles. The resulting concrete implementations have proven themselves in large numbers of applications. We suggest that it is imperative to thoroughly investigate which of these generative principles may be operative also in the brain, and hence relevant for cognitive neuroscience. In addition, ML research led to a range of interesting characterizations of neural information processing systems. We discuss five examples, the shortcomings of world modelling, the generation of thought processes, attention, neural scaling laws, and quantization, that illustrate how much neuroscience could potentially learn from ML research.