Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective
This addresses shape completion for 3D modeling applications, offering an unsupervised alternative to supervised methods, though it is incremental as it builds on deep prior concepts with NTK interpretation.
The paper tackles the problem of completing 3D shapes from incomplete scanned data by using an untrained neural network optimized per shape, interpreting this deep prior via neural tangent kernel theory to ensure completed patches resemble existing ones in kernel space. The result is an approach that completes large missing regions with plausible shapes without requiring extra training data.
We present a novel approach for completing and reconstructing 3D shapes from incomplete scanned data by using deep neural networks. Rather than being trained on supervised completion tasks and applied on a testing shape, the network is optimized from scratch on the single testing shape, to fully adapt to the shape and complete the missing data using contextual guidance from the known regions. The ability to complete missing data by an untrained neural network is usually referred to as the deep prior. In this paper, we interpret the deep prior from a neural tangent kernel (NTK) perspective and show that the completed shape patches by the trained CNN are naturally similar to existing patches, as they are proximate in the kernel feature space induced by NTK. The interpretation allows us to design more efficient network structures and learning mechanisms for the shape completion and reconstruction task. Being more aware of structural regularities than both traditional and other unsupervised learning-based reconstruction methods, our approach completes large missing regions with plausible shapes and complements supervised learning-based methods that use database priors by requiring no extra training data set and showing flexible adaptation to a particular shape instance.