Visual Question Generation for Class Acquisition of Unknown Objects
This addresses the challenge for computer vision systems to handle unknown objects in real-world scenarios, though it is an incremental step in interactive learning.
The paper tackles the problem of image recognition systems being limited to known object classes by proposing a method to generate questions about unknown objects in images, enabling acquisition of new class information from humans. Experimental results via human evaluation show the method successfully obtains information about unknown objects in an image dataset.
Traditional image recognition methods only consider objects belonging to already learned classes. However, since training a recognition model with every object class in the world is unfeasible, a way of getting information on unknown objects (i.e., objects whose class has not been learned) is necessary. A way for an image recognition system to learn new classes could be asking a human about objects that are unknown. In this paper, we propose a method for generating questions about unknown objects in an image, as means to get information about classes that have not been learned. Our method consists of a module for proposing objects, a module for identifying unknown objects, and a module for generating questions about unknown objects. The experimental results via human evaluation show that our method can successfully get information about unknown objects in an image dataset. Our code and dataset are available at https://github.com/mil-tokyo/vqg-unknown.