What Stops Learning-based 3D Registration from Working in the Real World?
It addresses the gap between synthetic benchmarks and real-world applications for 3D registration, offering incremental improvements to existing methods.
The paper tackled the problem of learning-based 3D point cloud registration failing on real-world data by identifying failure sources and proposing solutions, resulting in a best-practice network (BPNet) that achieves up to 67% accuracy on unseen real objects without fine-tuning.
Much progress has been made on the task of learning-based 3D point cloud registration, with existing methods yielding outstanding results on standard benchmarks, such as ModelNet40, even in the partial-to-partial matching scenario. Unfortunately, these methods still struggle in the presence of real data. In this work, we identify the sources of these failures, analyze the reasons behind them, and propose solutions to tackle them. We summarise our findings into a set of guidelines and demonstrate their effectiveness by applying them to different baseline methods, DCP and IDAM. In short, our guidelines improve both their training convergence and testing accuracy. Ultimately, this translates to a best-practice 3D registration network (BPNet), constituting the first learning-based method able to handle previously-unseen objects in real-world data. Despite being trained only on synthetic data, our model generalizes to real data without any fine-tuning, reaching an accuracy of up to 67% on point clouds of unseen objects obtained with a commercial sensor.