25 years of CNNs: Can we compare to human abstraction capabilities?
This addresses the problem of evaluating AI vision capabilities for researchers, showing incremental progress in abstract image classification.
The study compared LeNet and GoogLeNet over 25 years on classifying images by abstract properties, finding that CNNs still struggle with tasks humans solve easily, indicating limited progress in matching human abstraction.
We try to determine the progress made by convolutional neural networks over the past 25 years in classifying images into abstractc lasses. For this purpose we compare the performance of LeNet to that of GoogLeNet at classifying randomly generated images which are differentiated by an abstract property (e.g., one class contains two objects of the same size, the other class two objects of different sizes). Our results show that there is still work to do in order to solve vision problems humans are able to solve without much difficulty.