Reflecting After Learning for Understanding
This addresses the challenge for cognitive systems in making sense of conflicting classifier outputs, though it appears incremental as it builds on existing classification methods.
The paper tackles the problem of unifying multiple and competing predictions from image classifiers into abstractions, properties, or relationships, demonstrating that their framework unifies 41% to 46% of predictions on ImageNet and 51% on live video images, with higher rates when conceptual differences can be explained.
Today, image classification is a common way for systems to process visual content. Although neural network approaches to classification have seen great progress in reducing error rates, it is not clear what this means for a cognitive system that needs to make sense of the multiple and competing predictions from its own classifiers. As a step to address this, we present a novel framework that uses meta-reasoning and meta-operations to unify predictions into abstractions, properties, or relationships. Using the framework on images from ImageNet, we demonstrate systems that unify 41% to 46% of predictions in general and unify 67% to 75% of predictions when the systems can explain their conceptual differences. We also demonstrate a system in "the wild" by feeding live video images through it and show it unifying 51% of predictions in general and 69% of predictions when their differences can be explained conceptually by the system. In a survey given to 24 participants, we found that 87% of the unified predictions describe their corresponding images.