Silvia Sellán

Silvia Sellán, Yun-Chun Chen, Ziyi Wu et al. · gatech, nvidia

We introduce Breaking Bad, a large-scale dataset of fractured objects. Our dataset consists of over one million fractured objects simulated from ten thousand base models. The fracture simulation is powered by a recent physically based algorithm that efficiently generates a variety of fracture modes of an object. Existing shape assembly datasets decompose objects according to semantically meaningful parts, effectively modeling the construction process. In contrast, Breaking Bad models the destruction process of how a geometric object naturally breaks into fragments. Our dataset serves as a benchmark that enables the study of fractured object reassembly and presents new challenges for geometric shape understanding. We analyze our dataset with several geometry measurements and benchmark three state-of-the-art shape assembly deep learning methods under various settings. Extensive experimental results demonstrate the difficulty of our dataset, calling on future research in model designs specifically for the geometric shape assembly task. We host our dataset at https://breaking-bad-dataset.github.io/.

Lily Goli, Cody Reading, Silvia Sellán et al.

Neural Radiance Fields (NeRFs) have shown promise in applications like view synthesis and depth estimation, but learning from multiview images faces inherent uncertainties. Current methods to quantify them are either heuristic or computationally demanding. We introduce BayesRays, a post-hoc framework to evaluate uncertainty in any pre-trained NeRF without modifying the training process. Our method establishes a volumetric uncertainty field using spatial perturbations and a Bayesian Laplace approximation. We derive our algorithm statistically and show its superior performance in key metrics and applications. Additional results available at: https://bayesrays.github.io.

Silvia Sellán, Alec Jacobson

Reconstructing a surface from a point cloud is an underdetermined problem. We use a neural network to study and quantify this reconstruction uncertainty under a Poisson smoothness prior. Our algorithm addresses the main limitations of existing work and can be fully integrated into the 3D scanning pipeline, from obtaining an initial reconstruction to deciding on the next best sensor position and updating the reconstruction upon capturing more data.

Jingsen Zhu, Silvia Sellán, Alexander Terenin

We develop a framework for task-specific active next-best-view selection in 3D reconstruction from point clouds, by casting the problem in the language of Bayesian decision theory. Our framework works by (a) placing a prior distribution over the space of implicit surfaces, (b) using recently-developed stochastic surface reconstruction methods to calculate the resulting posterior distribution, then (c) using the posterior distribution to carefully reason about which view to scan next. This enables us to perform camera selection in a manner that is directly optimized for the intended use of the reconstructed data - meaning, we reduce uncertainty only in those regions that make a difference in the task at hand, as opposed to prior approaches that reduce it uniformly across space. We evaluate our method across three distinct downstream tasks: semantic classification, segmentation, and PDE-guided physics simulation. Experimental results demonstrate that our framework achieves superior task performance with fewer views compared to commonly used baselines and prior general uncertainty-reduction techniques.

Letao Chen, Sanju Mupparaju, Christopher Batty et al.

We propose a method to interpolate Signed Distance Function (SDF) data from a discrete set of samples. Unlike prior work, our approach ensures that the new SDF data values are fully consistent with the input and each other, such that the augmented data still corresponds to a geometrically realizable surface. We express the theoretical properties of SDFs as hard geometric constraints, and construct an efficient greedy algorithm for consistent SDF interpolation that is made even faster with powerful parallelized GPU preprocessing. We exemplify the usefulness of our method by evaluating it on three practical applications: global SDF refinement, in which the SDF data is upsampled without knowledge of the ground truth; mesh reconstruction, where our method can reconstruct highly detailed surfaces using global information from coarse input SDFs; and repair of pseudo-SDFs, which result from many pipelines such as CSG Boolean operations and must be turned into valid SDFs for downstream processing tasks. Our refined SDFs are guaranteed to be consistent with the input, where previous methods have no such guarantee.

Ningna Wang, Xiana Carrera, Christopher Batty et al.

Unsigned distance functions offer a powerful and flexible implicit surface representation that, unlike their signed counterparts, allow for surfaces that are open, non-orientable, or non-manifold. We consider the problem of reconstructing arbitrary surfaces from a finite set of samples of unsigned distance data. Existing methods for mesh reconstruction from distance data rely on sign information, accurate gradients, a corresponding continuous distance function, or extensive data-dependent training. However, they fail when applied to input that is both discrete and unsigned. Inspired by this challenge, we study the power diagram generated by the distance samples and propose a novel theoretical concept, the superpower contour, which we prove converges to the true surface in the limit of sampling density. We use this superpower contour as an initial surface proxy and design an algorithm that leverages it to produce a polygonal mesh approximating the unknown true geometry. Our method vastly outperforms other conceivable strategies for the discrete unsigned distance reconstruction task, and sets the stage for future work on this mathematically rich problem.

Sidhanth Holalkere, David S. Bindel, Silvia Sellán et al.

Poisson Surface Reconstruction is a widely-used algorithm for reconstructing a surface from an oriented point cloud. To facilitate applications where only partial surface information is available, or scanning is performed sequentially, a recent line of work proposes to incorporate uncertainty into the reconstructed surface via Gaussian process models. The resulting algorithms first perform Gaussian process interpolation, then solve a set of volumetric partial differential equations globally in space, resulting in a computationally expensive two-stage procedure. In this work, we apply recently-developed techniques from geometric Gaussian processes to combine interpolation and surface reconstruction into a single stage, requiring only one linear solve per sample. The resulting reconstructed surface samples can be queried locally in space, without the use of problem-dependent volumetric meshes or grids. These capabilities enable one to (a) perform probabilistic collision detection locally around the region of interest, (b) perform ray casting without evaluating points not on the ray's trajectory, and (c) perform next-view planning on a per-ray basis. They also do not requiring one to approximate kernel matrix inverses with diagonal matrices as part of intermediate computations, unlike prior methods. Results show that our approach provides a cleaner, more-principled, and more-flexible stochastic surface reconstruction pipeline.