Understanding Attention: In Minds and Machines
This paper aims to unify the understanding of attention across AI and Neuroscience, which is a foundational problem for researchers in both fields.
This paper reviews the concept and variants of attention in artificial neural networks (ANNs) and discusses its origins from a neuroscience perspective. It suggests grounding ideas on common conceptual frameworks for a systematic analysis and potential unification of attention concepts in AI and Neuroscience.
Attention is a complex and broad concept, studied across multiple disciplines spanning artificial intelligence, cognitive science, psychology, neuroscience, and related fields. Although many of the ideas regarding attention do not significantly overlap among these fields, there is a common theme of adaptive control of limited resources. In this work, we review the concept and variants of attention in artificial neural networks (ANNs). We also discuss the origin of attention from the neuroscience point of view parallel to that of ANNs. Instead of having seemingly disconnected dialogues between varied disciplines, we suggest grounding the ideas on common conceptual frameworks for a systematic analysis of attention and towards possible unification of ideas in AI and Neuroscience.