K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition
This work addresses the gap in visual question generation for knowledge acquisition, which is incremental as it introduces a new dataset and model for a specific domain.
The authors tackled the problem of generating questions from images with a focus on knowledge acquisition, constructing the first large human-annotated dataset (K-VQG) linking questions to structured knowledge and developing a model that outperforms existing ones on this dataset.
Visual Question Generation (VQG) is a task to generate questions from images. When humans ask questions about an image, their goal is often to acquire some new knowledge. However, existing studies on VQG have mainly addressed question generation from answers or question categories, overlooking the objectives of knowledge acquisition. To introduce a knowledge acquisition perspective into VQG, we constructed a novel knowledge-aware VQG dataset called K-VQG. This is the first large, humanly annotated dataset in which questions regarding images are tied to structured knowledge. We also developed a new VQG model that can encode and use knowledge as the target for a question. The experiment results show that our model outperforms existing models on the K-VQG dataset.