Joint Supervised and Self-Supervised Learning for 3D Real-World Challenges
This work addresses challenges in 3D shape understanding for AI agents interacting with real-world environments, though it appears incremental in its approach.
The paper tackled the problem of 3D point cloud processing under data scarcity and domain gaps by proposing a multi-task model that combines supervised and self-supervised learning, achieving state-of-the-art results in shape classification and part segmentation.
Point cloud processing and 3D shape understanding are very challenging tasks for which deep learning techniques have demonstrated great potentials. Still further progresses are essential to allow artificial intelligent agents to interact with the real world, where the amount of annotated data may be limited and integrating new sources of knowledge becomes crucial to support autonomous learning. Here we consider several possible scenarios involving synthetic and real-world point clouds where supervised learning fails due to data scarcity and large domain gaps. We propose to enrich standard feature representations by leveraging self-supervision through a multi-task model that can solve a 3D puzzle while learning the main task of shape classification or part segmentation. An extensive analysis investigating few-shot, transfer learning and cross-domain settings shows the effectiveness of our approach with state-of-the-art results for 3D shape classification and part segmentation.