A Dataset for Medical Instructional Video Classification and Question Answering
This work addresses the need for cross-modal understanding in medical videos to benefit the public and practitioners, but it is incremental as it focuses on dataset creation and benchmarking.
The paper tackles the problem of understanding medical videos by introducing two new datasets, MedVidCL and MedVidQA, for classification and question answering tasks, with results including 6,117 annotated videos for classification and 3,010 annotated questions for answer localization, setting competitive baselines.
This paper introduces a new challenge and datasets to foster research toward designing systems that can understand medical videos and provide visual answers to natural language questions. We believe medical videos may provide the best possible answers to many first aids, medical emergency, and medical education questions. Toward this, we created the MedVidCL and MedVidQA datasets and introduce the tasks of Medical Video Classification (MVC) and Medical Visual Answer Localization (MVAL), two tasks that focus on cross-modal (medical language and medical video) understanding. The proposed tasks and datasets have the potential to support the development of sophisticated downstream applications that can benefit the public and medical practitioners. Our datasets consist of 6,117 annotated videos for the MVC task and 3,010 annotated questions and answers timestamps from 899 videos for the MVAL task. These datasets have been verified and corrected by medical informatics experts. We have also benchmarked each task with the created MedVidCL and MedVidQA datasets and proposed the multimodal learning methods that set competitive baselines for future research.