Visual analogy: Deep learning versus compositional models
This addresses the problem of understanding analogical reasoning mechanisms for cognitive science and AI, but it is incremental as it builds on existing model comparisons.
The study compared human performance on visual analogies with deep learning and compositional models, finding that only the compositional model based on part representations matched human qualitative performance, while deep learning models failed to do so.
Is analogical reasoning a task that must be learned to solve from scratch by applying deep learning models to massive numbers of reasoning problems? Or are analogies solved by computing similarities between structured representations of analogs? We address this question by comparing human performance on visual analogies created using images of familiar three-dimensional objects (cars and their subregions) with the performance of alternative computational models. Human reasoners achieved above-chance accuracy for all problem types, but made more errors in several conditions (e.g., when relevant subregions were occluded). We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations. The compositional model based on part representations, but not the deep learning models, generated qualitative performance similar to that of human reasoners.