Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
This work addresses the problem of classifying objects in 3D point clouds without labeled data, which is incremental as it adapts an existing model to a new domain.
The paper tackles zero-shot point cloud classification by leveraging GPT-4 Vision to overcome limitations of prior methods like PointCLIP, achieving superior performance and setting a new benchmark in this area.
In this study, we tackle the challenge of classifying the object category in point clouds, which previous works like PointCLIP struggle to address due to the inherent limitations of the CLIP architecture. Our approach leverages GPT-4 Vision (GPT-4V) to overcome these challenges by employing its advanced generative abilities, enabling a more adaptive and robust classification process. We adapt the application of GPT-4V to process complex 3D data, enabling it to achieve zero-shot recognition capabilities without altering the underlying model architecture. Our methodology also includes a systematic strategy for point cloud image visualization, mitigating domain gap and enhancing GPT-4V's efficiency. Experimental validation demonstrates our approach's superiority in diverse scenarios, setting a new benchmark in zero-shot point cloud classification.