MiXR: Harvesting and Recomposing Geometry from Real-World Objects for In-Situ 3D Design
For 3D designers and hobbyists, MiXR provides a novel hybrid workflow that combines explicit spatial control with generative refinement, addressing the lack of precise geometric composition in current text/image-based 3D generation.
MiXR is an XR system that lets users harvest geometry from real-world objects and recompose it into new 3D models, with generative AI refining the final shape. In a user study (N=12), participants rated their designs as significantly closer to the target, felt more in control, and experienced lower cognitive workload compared to a generative baseline.
Recent developments in 3D generative AI enable users to create bespoke 3D models from text or image prompts. However, these approaches provide limited control over spatial structure, making them ill suited for tasks requiring precise geometric composition. We present MiXR, an XR system for in-situ compositional modeling that enables users to create new 3D models by harvesting geometry from their environment. Users extract segments from captured objects and assemble new artifacts through direct 3D manipulation, while generative AI synthesizes a coherent model from the user-defined composition. This hybrid workflow allows users to define spatial structure explicitly while delegating geometric refinement to generative models, enabling them to specify spatial intent that is difficult to express through verbal prompts alone. In a controlled user study ($N=12$), participants using MiXR rated their designs as significantly closer to the target, felt more in control, and experienced lower cognitive workload compared to a generative composition baseline.