Scene Synthesis via Uncertainty-Driven Attribute Synchronization
This work addresses scene synthesis for applications in architectural CAD, computer graphics, and virtual robot training environments, representing an incremental improvement over existing methods.
The paper tackles the problem of generating 3D scenes with diverse patterns by introducing a neural synthesis approach that uses uncertainty-driven attribute synchronization and over-complete predictions to prune infeasible outputs, resulting in outperforming existing methods and faithfully interpolating training data while preserving continuous and discrete patterns.
Developing deep neural networks to generate 3D scenes is a fundamental problem in neural synthesis with immediate applications in architectural CAD, computer graphics, as well as in generating virtual robot training environments. This task is challenging because 3D scenes exhibit diverse patterns, ranging from continuous ones, such as object sizes and the relative poses between pairs of shapes, to discrete patterns, such as occurrence and co-occurrence of objects with symmetrical relationships. This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches. We use the parametric prior distributions learned from training data, which provide uncertainties of object attributes and relative attributes, to regularize the outputs of feed-forward neural models. Moreover, instead of merely predicting a scene layout, our approach predicts an over-complete set of attributes. This methodology allows us to utilize the underlying consistency constraints among the predicted attributes to prune infeasible predictions. Experimental results show that our approach outperforms existing methods considerably. The generated 3D scenes interpolate the training data faithfully while preserving both continuous and discrete feature patterns.