Metaphors We Learn By
This is an incremental conceptual essay that reframes existing AI techniques through a cognitive metaphor lens, potentially influencing theoretical discussions in AI and cognitive science.
The paper explores the connection between parameter sharing in neural networks and analogy making in cognitive science, proposing that this perspective can bridge connectionist and rule-based computational views.
Gradient based learning using error back-propagation (``backprop'') is a well-known contributor to much of the recent progress in AI. A less obvious, but arguably equally important, ingredient is parameter sharing - most well-known in the context of convolutional networks. In this essay we relate parameter sharing (``weight sharing'') to analogy making and the school of thought of cognitive metaphor. We discuss how recurrent and auto-regressive models can be thought of as extending analogy making from static features to dynamic skills and procedures. We also discuss corollaries of this perspective, for example, how it can challenge the currently entrenched dichotomy between connectionist and ``classic'' rule-based views of computation.