Generic Event Boundary Detection in Video with Pyramid Features
This work addresses the problem of automatically segmenting videos into meaningful events for applications like video analysis and summarization, representing an incremental improvement over existing methods.
The paper tackles generic event boundary detection in video by using pyramid features to capture spatial and temporal correlations between frames, achieving state-of-the-art performance on the GEBD benchmark dataset and demonstrating effectiveness on the TAPOS dataset.
Generic event boundary detection (GEBD) aims to split video into chunks at a broad and diverse set of actions as humans naturally perceive event boundaries. In this study, we present an approach that considers the correlation between neighbor frames with pyramid feature maps in both spatial and temporal dimensions to construct a framework for localizing generic events in video. The features at multiple spatial dimensions of a pre-trained ResNet-50 are exploited with different views in the temporal dimension to form a temporal pyramid feature map. Based on that, the similarity between neighbor frames is calculated and projected to build a temporal pyramid similarity feature vector. A decoder with 1D convolution operations is used to decode these similarities to a new representation that incorporates their temporal relationship for later boundary score estimation. Extensive experiments conducted on the GEBD benchmark dataset show the effectiveness of our system and its variations, in which we outperformed the state-of-the-art approaches. Additional experiments on TAPOS dataset, which contains long-form videos with Olympic sport actions, demonstrated the effectiveness of our study compared to others.