Gaussian Building Mesh (GBM): Extract a Building's 3D Mesh with Google Earth and Gaussian Splatting
This provides a tool for urban planning, architecture, or GIS applications by enabling automated 3D modeling of buildings, though it is incremental as it integrates existing methods with minor improvements.
The paper tackles the problem of extracting 3D meshes of buildings from 2D images by combining Google Earth Studio, SAM2+GroundingDINO, and Gaussian Splatting with mask refinement techniques, resulting in a pipeline that can generate 3D meshes based on text or location inputs without labeled training data.
Recently released open-source pre-trained foundational image segmentation and object detection models (SAM2+GroundingDINO) allow for geometrically consistent segmentation of objects of interest in multi-view 2D images. Users can use text-based or click-based prompts to segment objects of interest without requiring labeled training datasets. Gaussian Splatting allows for the learning of the 3D representation of a scene's geometry and radiance based on 2D images. Combining Google Earth Studio, SAM2+GroundingDINO, 2D Gaussian Splatting, and our improvements in mask refinement based on morphological operations and contour simplification, we created a pipeline to extract the 3D mesh of any building based on its name, address, or geographic coordinates.