Challenge of Spatial Cognition for Deep Learning
This work highlights a fundamental limitation in deep learning for spatial cognition, cautioning that without manual priors, DCNNs may fail at basic tasks, which is incremental as it builds on known bottlenecks in generalization.
The paper tackles the problem of whether deep convolutional neural networks (DCNNs) can learn spatial concepts like straightness and convexity from visual examples, finding that standard data-driven deep learning fails due to superficial variations, but by incorporating task-specific convolutional kernels, they achieve generalization to out-of-distribution images.
Given the success of the deep convolutional neural networks (DCNNs) in applications of visual recognition and classification, it would be tantalizing to test if DCNNs can also learn spatial concepts, such as straightness, convexity, left/right, front/back, relative size, aspect ratio, polygons, etc., from varied visual examples of these concepts that are simple and yet vital for spatial reasoning. Much to our dismay, extensive experiments of the type of cognitive psychology demonstrate that the data-driven deep learning (DL) cannot see through superficial variations in visual representations and grasp the spatial concept in abstraction. The root cause of failure turns out to be the learning methodology, not the computational model of the neural network itself. By incorporating task-specific convolutional kernels, we are able to construct DCNNs for spatial cognition tasks that can generalize to input images not drawn from the same distribution of the training set. This work raises a precaution that without manually-incorporated priors or features DCCNs may fail spatial cognitive tasks at rudimentary level.