Masked Autoencoder for Unsupervised Video Summarization
This provides a self-supervised method for video summarization that is robust and flexible, though it is incremental as it builds on existing autoencoder techniques.
The authors tackled unsupervised video summarization by proposing a masked autoencoder that uses reconstruction scores to evaluate frame importance, achieving competitive results on major benchmarks without extra downstream design or fine-tuning.
Summarizing a video requires a diverse understanding of the video, ranging from recognizing scenes to evaluating how much each frame is essential enough to be selected as a summary. Self-supervised learning (SSL) is acknowledged for its robustness and flexibility to multiple downstream tasks, but the video SSL has not shown its value for dense understanding tasks like video summarization. We claim an unsupervised autoencoder with sufficient self-supervised learning does not need any extra downstream architecture design or fine-tuning weights to be utilized as a video summarization model. The proposed method to evaluate the importance score of each frame takes advantage of the reconstruction score of the autoencoder's decoder. We evaluate the method in major unsupervised video summarization benchmarks to show its effectiveness under various experimental settings.