Towards Training-Free Open-World Classification with 3D Generative Models
This addresses the problem of open-category and pose-invariant recognition in dynamic 3D environments, offering a training-free approach that is incremental over prior 2D-based methods.
The paper tackles 3D open-world classification by using 3D generative models to avoid reliance on 2D projections, achieving state-of-the-art performance with 32.0% and 8.7% overall accuracy improvements on ModelNet10 and McGill datasets.
3D open-world classification is a challenging yet essential task in dynamic and unstructured real-world scenarios, requiring both open-category and open-pose recognition. To address these challenges, recent wisdom often takes sophisticated 2D pre-trained models to provide enriched and stable representations. However, these methods largely rely on how 3D objects can be projected into 2D space, which is unfortunately not well solved, and thus significantly limits their performance. Unlike these present efforts, in this paper we make a pioneering exploration of 3D generative models for 3D open-world classification. Drawing on abundant prior knowledge from 3D generative models, we additionally craft a rotation-invariant feature extractor. This innovative synergy endows our pipeline with the advantages of being training-free, open-category, and pose-invariant, thus well suited to 3D open-world classification. Extensive experiments on benchmark datasets demonstrate the potential of generative models in 3D open-world classification, achieving state-of-the-art performance on ModelNet10 and McGill with 32.0% and 8.7% overall accuracy improvement, respectively.