Detection Bank: An Object Detection Based Video Representation for Multimedia Event Recognition
This work addresses the challenge of capturing complex semantics in videos for event recognition, offering an incremental improvement over existing methods.
The paper tackles the problem of representing videos for multimedia event recognition by proposing Detection Bank, a representation based on object detections, which when combined with existing descriptors significantly outperforms them alone on the TRECVID MED 2011 dataset.
While low-level image features have proven to be effective representations for visual recognition tasks such as object recognition and scene classification, they are inadequate to capture complex semantic meaning required to solve high-level visual tasks such as multimedia event detection and recognition. Recognition or retrieval of events and activities can be improved if specific discriminative objects are detected in a video sequence. In this paper, we propose an image representation, called Detection Bank, based on the detection images from a large number of windowed object detectors where an image is represented by different statistics derived from these detections. This representation is extended to video by aggregating the key frame level image representations through mean and max pooling. We empirically show that it captures complementary information to state-of-the-art representations such as Spatial Pyramid Matching and Object Bank. These descriptors combined with our Detection Bank representation significantly outperforms any of the representations alone on TRECVID MED 2011 data.