DenseDINO: Boosting Dense Self-Supervised Learning with Token-Based Point-Level Consistency
This work addresses the need for better dense prediction tasks in computer vision, though it appears incremental as it builds on existing self-supervised transformers.
The paper tackled the problem of learning dense visual representations in self-supervised learning by introducing DenseDINO, a transformer framework that uses token-based point-level consistency to exploit spatial information, resulting in a 7.2% mIoU improvement in semantic segmentation on PascalVOC.
In this paper, we propose a simple yet effective transformer framework for self-supervised learning called DenseDINO to learn dense visual representations. To exploit the spatial information that the dense prediction tasks require but neglected by the existing self-supervised transformers, we introduce point-level supervision across views in a novel token-based way. Specifically, DenseDINO introduces some extra input tokens called reference tokens to match the point-level features with the position prior. With the reference token, the model could maintain spatial consistency and deal with multi-object complex scene images, thus generalizing better on dense prediction tasks. Compared with the vanilla DINO, our approach obtains competitive performance when evaluated on classification in ImageNet and achieves a large margin (+7.2% mIoU) improvement in semantic segmentation on PascalVOC under the linear probing protocol for segmentation.