Understanding Foundation Models: Are We Back in 1924?
It addresses the problem of interpreting AI model capabilities for researchers, noting that comparisons to neuroscience are limited and understanding remains incremental.
This position paper examines Foundation Models (FMs) in AI, arguing that their reasoning improvements stem from novel training techniques like grokking rather than increased model size, while highlighting challenges in benchmarking and understanding their inner workings.
This position paper explores the rapid development of Foundation Models (FMs) in AI and their implications for intelligence and reasoning. It examines the characteristics of FMs, including their training on vast datasets and use of embedding spaces to capture semantic relationships. The paper discusses recent advancements in FMs' reasoning abilities which we argue cannot be attributed to increased model size but to novel training techniques which yield learning phenomena like grokking. It also addresses the challenges in benchmarking FMs and compares their structure to the human brain. We argue that while FMs show promising developments in reasoning and knowledge representation, understanding their inner workings remains a significant challenge, similar to ongoing efforts in neuroscience to comprehend human brain function. Despite having some similarities, fundamental differences between FMs and the structure of human brain warn us against making direct comparisons or expecting neuroscience to provide immediate insights into FM function.