From Cognition to Computation: A Comparative Review of Human Attention and Transformer Architectures
It provides an interdisciplinary analysis for researchers aiming to improve AI generalization by drawing from human cognition, but it is incremental as a review.
This review compares human attention and Transformer architectures, highlighting differences in capacity constraints, attention pathways, and intentional mechanisms to address open research questions in AI.
Attention is a cornerstone of human cognition that facilitates the efficient extraction of information in everyday life. Recent developments in artificial intelligence like the Transformer architecture also incorporate the idea of attention in model designs. However, despite the shared fundamental principle of selectively attending to information, human attention and the Transformer model display notable differences, particularly in their capacity constraints, attention pathways, and intentional mechanisms. Our review aims to provide a comparative analysis of these mechanisms from a cognitive-functional perspective, thereby shedding light on several open research questions. The exploration encourages interdisciplinary efforts to derive insights from human attention mechanisms in the pursuit of developing more generalized artificial intelligence.