ObitoNet: Multimodal High-Resolution Point Cloud Reconstruction
This work addresses the problem of generating detailed 3D point clouds for applications in computer vision and robotics, presenting an incremental improvement through a novel hybrid method.
The paper tackles high-resolution point cloud reconstruction from multimodal inputs by integrating image and geometric data using a Cross Attention mechanism, achieving robust generation in challenging conditions like sparse or noisy data.
ObitoNet employs a Cross Attention mechanism to integrate multimodal inputs, where Vision Transformers (ViT) extract semantic features from images and a point cloud tokenizer processes geometric information using Farthest Point Sampling (FPS) and K Nearest Neighbors (KNN) for spatial structure capture. The learned multimodal features are fed into a transformer-based decoder for high-resolution point cloud reconstruction. This approach leverages the complementary strengths of both modalities rich image features and precise geometric details ensuring robust point cloud generation even in challenging conditions such as sparse or noisy data.