Long-Term Feature Banks for Detailed Video Understanding
This addresses the challenge of detailed video understanding for computer vision applications, representing an incremental improvement by augmenting existing models.
The paper tackled the problem of enabling video models to incorporate long-term context by proposing a long-term feature bank that extracts supportive information from entire videos, achieving state-of-the-art results on AVA, EPIC-Kitchens, and Charades datasets.
To understand the world, we humans constantly need to relate the present to the past, and put events in context. In this paper, we enable existing video models to do the same. We propose a long-term feature bank---supportive information extracted over the entire span of a video---to augment state-of-the-art video models that otherwise would only view short clips of 2-5 seconds. Our experiments demonstrate that augmenting 3D convolutional networks with a long-term feature bank yields state-of-the-art results on three challenging video datasets: AVA, EPIC-Kitchens, and Charades.