SoilNet: A Multimodal Multitask Model for Hierarchical Classification of Soil Horizons
This work addresses soil horizon classification for monitoring soil health, impacting agriculture, food security, and climate resilience, but it is incremental as it applies existing multimodal and multitask techniques to a specific domain.
The authors tackled soil horizon classification, which is challenging due to multimodal data and a complex hierarchical label structure, by proposing SoilNet, a multimodal multitask model that integrates image data and geotemporal metadata to predict depth markers, segment soil profiles, and classify horizons using a graph-based label representation, achieving improved accuracy on a real-world dataset.
While recent advances in foundation models have improved the state of the art in many domains, some problems in empirical sciences could not benefit from this progress yet. Soil horizon classification, for instance, remains challenging because of its multimodal and multitask characteristics and a complex hierarchically structured label taxonomy. Accurate classification of soil horizons is crucial for monitoring soil health, which directly impacts agricultural productivity, food security, ecosystem stability and climate resilience. In this work, we propose $\textit{SoilNet}$ - a multimodal multitask model to tackle this problem through a structured modularized pipeline. Our approach integrates image data and geotemporal metadata to first predict depth markers, segmenting the soil profile into horizon candidates. Each segment is characterized by a set of horizon-specific morphological features. Finally, horizon labels are predicted based on the multimodal concatenated feature vector, leveraging a graph-based label representation to account for the complex hierarchical relationships among soil horizons. Our method is designed to address complex hierarchical classification, where the number of possible labels is very large, imbalanced and non-trivially structured. We demonstrate the effectiveness of our approach on a real-world soil profile dataset. All code and experiments can be found in our repository: https://github.com/calgo-lab/BGR/