MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation
It addresses efficient video object segmentation for resource-constrained devices like mobile phones, representing an incremental improvement by optimizing existing methods for speed and size.
This paper tackles real-time video object segmentation on mobile devices by combining knowledge distillation with contrastive learning, achieving competitive accuracy on DAVIS and YouTube benchmarks while running up to 5x faster with 32x fewer parameters and 32 milliseconds per frame on a Samsung Galaxy S22.
This paper tackles the problem of semi-supervised video object segmentation on resource-constrained devices, such as mobile phones. We formulate this problem as a distillation task, whereby we demonstrate that small space-time-memory networks with finite memory can achieve competitive results with state of the art, but at a fraction of the computational cost (32 milliseconds per frame on a Samsung Galaxy S22). Specifically, we provide a theoretically grounded framework that unifies knowledge distillation with supervised contrastive representation learning. These models are able to jointly benefit from both pixel-wise contrastive learning and distillation from a pre-trained teacher. We validate this loss by achieving competitive J&F to state of the art on both the standard DAVIS and YouTube benchmarks, despite running up to 5x faster, and with 32x fewer parameters.