WhyAct: Identifying Action Reasons in Lifestyle Vlogs
This addresses the challenge of understanding action motivations in online videos for applications in video analysis and AI comprehension, representing an incremental advance with a new dataset.
The paper tackles the problem of automatically identifying human action reasons in lifestyle vlogs by introducing the WhyAct dataset with 1,077 annotated actions and developing a multimodal model that uses visual and textual information to infer reasons.
We aim to automatically identify human action reasons in online videos. We focus on the widespread genre of lifestyle vlogs, in which people perform actions while verbally describing them. We introduce and make publicly available the WhyAct dataset, consisting of 1,077 visual actions manually annotated with their reasons. We describe a multimodal model that leverages visual and textual information to automatically infer the reasons corresponding to an action presented in the video.