CropCraft: Inverse Procedural Modeling for 3D Reconstruction of Crop Plants
This addresses the need for accurate 3D digital twins of plants in agriculture, environmental science, and robotics, representing a novel method for a known bottleneck.
The paper tackles the problem of incomplete 3D reconstruction of crop plants from images due to occlusion and complex geometries by introducing an inverse procedural modeling method that optimizes plant morphology parameters, resulting in complete and biologically plausible 3D models validated on real agricultural field images.
The ability to automatically build 3D digital twins of plants from images has countless applications in agriculture, environmental science, robotics, and other fields. However, current 3D reconstruction methods fail to recover complete shapes of plants due to heavy occlusion and complex geometries. In this work, we present a novel method for 3D reconstruction of agricultural crops based on optimizing a parametric model of plant morphology via inverse procedural modeling. Our method first estimates depth maps by fitting a neural radiance field and then employs Bayesian optimization to estimate plant morphological parameters that result in consistent depth renderings. The resulting 3D model is complete and biologically plausible. We validate our method on a dataset of real images of agricultural fields, and demonstrate that the reconstructions can be used for a variety of monitoring and simulation applications.