Classifying geospatial objects from multiview aerial imagery using semantic meshes
This addresses the need for more accurate and robust object classification in Earth science and natural resource management by capturing vertical aspects and multiple viewpoints, though it is incremental as it builds on existing multiview and mesh techniques.
The paper tackled the problem of classifying geospatial objects like tree species from aerial imagery by moving beyond single top-down orthomosaics to use raw multiview images with semantic meshes, resulting in an accuracy improvement from 53% to 75% on a cross-site classification task.
Aerial imagery is increasingly used in Earth science and natural resource management as a complement to labor-intensive ground-based surveys. Aerial systems can collect overlapping images that provide multiple views of each location from different perspectives. However, most prediction approaches (e.g. for tree species classification) use a single, synthesized top-down "orthomosaic" image as input that contains little to no information about the vertical aspects of objects and may include processing artifacts. We propose an alternate approach that generates predictions directly on the raw images and accurately maps these predictions into geospatial coordinates using semantic meshes. This method$\unicode{x2013}$released as a user-friendly open-source toolkit$\unicode{x2013}$enables analysts to use the highest quality data for predictions, capture information about the sides of objects, and leverage multiple viewpoints of each location for added robustness. We demonstrate the value of this approach on a new benchmark dataset of four forest sites in the western U.S. that consists of drone images, photogrammetry results, predicted tree locations, and species classification data derived from manual surveys. We show that our proposed multiview method improves classification accuracy from 53% to 75% relative to an orthomosaic baseline on a challenging cross-site tree species classification task.