Relation-Aware Pyramid Network (RapNet) for temporal action proposal
This is an incremental improvement for video action detection researchers, focusing on a specific competition task.
The authors tackled temporal action proposal generation by fine-tuning a ResNet-50-C3D CNN on ActivityNet v1.3 and designing a Relation-Aware Pyramid Network (RapNet) with a two-stage boundary adjustment scheme, achieving 2nd place in the ActivityNet Challenge 2019.
In this technical report, we describe our solution to temporal action proposal (task 1) in ActivityNet Challenge 2019. First, we fine-tune a ResNet-50-C3D CNN on ActivityNet v1.3 based on Kinetics pretrained model to extract snippet-level video representations and then we design a Relation-Aware Pyramid Network (RapNet) to generate temporal multiscale proposals with confidence score. After that, we employ a two-stage snippet-level boundary adjustment scheme to re-rank the order of generated proposals. Ensemble methods are also been used to improve the performance of our solution, which helps us achieve 2nd place.