All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams
For process analysts, SnapLog bridges the gap between video data and event logs, enabling automated process discovery from a previously inaccessible data modality.
SnapLog extracts event data from videos by converting frames to feature vectors, performing temporal segmentation via similarity matrices, and using generalized few-shot classification to label segments, enabling process mining on video streams. The approach produces logs that accurately reflect the underlying process.
Disciplines such as business process management and process mining aid organizations by discovering insights about processes on the basis of recorded event data. However, an obstacle to process analysis is data multi-modality: for instance, data in video form are not directly interpretable as events. In this work, we present SnapLog, an approach to extract event data from videos by converting frames to feature vectors using image embeddings and performing temporal segmentation through frame-wise similarity matrices. A generalized few-shot classification is then used to assign labels to the video segments, yielding labeled, timestamped sub-sequences of frames that are interpretable as events. Conventional process mining techniques can be used to analyze the resulting data. We show that our approach produces logs that accurately reflect the process in the videos.