DOS: Distilling Observable Softmaps of Zipfian Prototypes for Self-Supervised Point Representation
This work addresses the problem of learning robust 3D point cloud representations without annotations for applications like autonomous driving and robotics, offering a scalable paradigm with incremental improvements over existing methods.
The paper tackles challenges in self-supervised learning for 3D point clouds, such as irregular geometry and unbalanced semantics, by proposing DOS, a framework that distills semantic relevance softmaps at observable points and uses Zipfian prototypes; it achieves state-of-the-art performance on semantic segmentation and 3D object detection across multiple benchmarks.
Recent advances in self-supervised learning (SSL) have shown tremendous potential for learning 3D point cloud representations without human annotations. However, SSL for 3D point clouds still faces critical challenges due to irregular geometry, shortcut-prone reconstruction, and unbalanced semantics distribution. In this work, we propose DOS (Distilling Observable Softmaps), a novel SSL framework that self-distills semantic relevance softmaps only at observable (unmasked) points. This strategy prevents information leakage from masked regions and provides richer supervision than discrete token-to-prototype assignments. To address the challenge of unbalanced semantics in an unsupervised setting, we introduce Zipfian prototypes and incorporate them using a modified Sinkhorn-Knopp algorithm, Zipf-Sinkhorn, which enforces a power-law prior over prototype usage and modulates the sharpness of the target softmap during training. DOS outperforms current state-of-the-art methods on semantic segmentation and 3D object detection across multiple benchmarks, including nuScenes, Waymo, SemanticKITTI, ScanNet, and ScanNet200, without relying on extra data or annotations. Our results demonstrate that observable-point softmaps distillation offers a scalable and effective paradigm for learning robust 3D representations.