Ladder Siamese Network: a Method and Insights for Multi-level Self-Supervised Learning
This addresses training issues in self-supervised learning for computer vision, offering incremental improvements across multiple tasks.
The paper tackles slow convergence and instability in Siamese-network-based self-supervised learning by proposing the Ladder Siamese Network, which uses intermediate self-supervisions to improve training; it shows improvements of 1.0% in ImageNet classification, 1.2% in COCO detection, and 3.1% in PASCAL VOC segmentation over baselines.
Siamese-network-based self-supervised learning (SSL) suffers from slow convergence and instability in training. To alleviate this, we propose a framework to exploit intermediate self-supervisions in each stage of deep nets, called the Ladder Siamese Network. Our self-supervised losses encourage the intermediate layers to be consistent with different data augmentations to single samples, which facilitates training progress and enhances the discriminative ability of the intermediate layers themselves. While some existing work has already utilized multi-level self supervisions in SSL, ours is different in that 1) we reveal its usefulness with non-contrastive Siamese frameworks in both theoretical and empirical viewpoints, and 2) ours improves image-level classification, instance-level detection, and pixel-level segmentation simultaneously. Experiments show that the proposed framework can improve BYOL baselines by 1.0% points in ImageNet linear classification, 1.2% points in COCO detection, and 3.1% points in PASCAL VOC segmentation. In comparison with the state-of-the-art methods, our Ladder-based model achieves competitive and balanced performances in all tested benchmarks without causing large degradation in one.