Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms
This addresses the multimodality challenge in community Q&A platforms, which is an incremental improvement for enhancing information organization and accessibility.
The paper tackled the problem of extending question categorization and expert retrieval methods to handle image-based questions in community Q&A platforms, by adapting visual question answering models, resulting in a final model that markedly outperformed text-only and VQA baselines on real-world data.
Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only the text modality. With the increasing multimodal nature of web content, we focus on extending these methods for CQA questions accompanied by images. Specifically, we leverage the success of representation learning for text and images in the visual question answering (VQA) domain, and adapt the underlying concept and architecture for automated category classification and expert retrieval on image-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of Yahoo! Answers. To the best of our knowledge, this is the first work to tackle the multimodality challenge in CQA, and to adapt VQA models for tasks on a more ecologically valid source of visual questions. Our analysis of the differences between visual QA and community QA data drives our proposal of novel augmentations of an attention method tailored for CQA, and use of auxiliary tasks for learning better grounding features. Our final model markedly outperforms the text-only and VQA model baselines for both tasks of classification and expert retrieval on real-world multimodal CQA data.