COALESCE: Component Assembly by Learning to Synthesize Connections
This addresses the challenge of creating coherent 3D objects from diverse parts for applications in computer graphics and design, representing a novel method for a known bottleneck rather than an incremental improvement.
The paper tackles the problem of assembling 3D shapes from mismatched components by introducing COALESCE, a data-driven framework that uses deep learning to synthesize plausible part connections, and demonstrates that it significantly outperforms prior methods including state-of-the-art shape completion approaches.
We introduce COALESCE, the first data-driven framework for component-based shape assembly which employs deep learning to synthesize part connections. To handle geometric and topological mismatches between parts, we remove the mismatched portions via erosion, and rely on a joint synthesis step, which is learned from data, to fill the gap and arrive at a natural and plausible part joint. Given a set of input parts extracted from different objects, COALESCE automatically aligns them and synthesizes plausible joints to connect the parts into a coherent 3D object represented by a mesh. The joint synthesis network, designed to focus on joint regions, reconstructs the surface between the parts by predicting an implicit shape representation that agrees with existing parts, while generating a smooth and topologically meaningful connection. We employ test-time optimization to further ensure that the synthesized joint region closely aligns with the input parts to create realistic component assemblies from diverse input parts. We demonstrate that our method significantly outperforms prior approaches including baseline deep models for 3D shape synthesis, as well as state-of-the-art methods for shape completion.