Text To 3D Object Generation For Scalable Room Assembly
This addresses the data scarcity problem for researchers and developers in computer vision and machine learning, though it appears incremental as it builds on existing methods like text-to-image and Neural Radiance Fields.
The paper tackles the problem of data scarcity for scene understanding models by proposing an end-to-end system that generates synthetic 3D indoor scenes from text prompts, resulting in scalable and customizable high-fidelity data to improve model robustness and generalizability.
Modern machine learning models for scene understanding, such as depth estimation and object tracking, rely on large, high-quality datasets that mimic real-world deployment scenarios. To address data scarcity, we propose an end-to-end system for synthetic data generation for scalable, high-quality, and customizable 3D indoor scenes. By integrating and adapting text-to-image and multi-view diffusion models with Neural Radiance Field-based meshing, this system generates highfidelity 3D object assets from text prompts and incorporates them into pre-defined floor plans using a rendering tool. By introducing novel loss functions and training strategies into existing methods, the system supports on-demand scene generation, aiming to alleviate the scarcity of current available data, generally manually crafted by artists. This system advances the role of synthetic data in addressing machine learning training limitations, enabling more robust and generalizable models for real-world applications.