A Multimodal Conversational Assistant for the Characterization of Agricultural Plots from Geospatial Open Data
This work addresses the accessibility issue for non-expert users in sustainable land management by lowering the barrier to specialized agricultural information through natural language interaction, though it is incremental in combining existing methods like RAG and LLMs.
This study tackled the problem of high technical barriers for non-experts accessing agricultural and geospatial data by developing a multimodal conversational assistant that integrates retrieval-augmented generation with large language models, resulting in a system capable of generating clear, relevant, and context-aware responses to agricultural queries while being reproducible and scalable.
The increasing availability of open Earth Observation (EO) and agricultural datasets holds great potential for supporting sustainable land management. However, their high technical entry barrier limits accessibility for non-expert users. This study presents an open-source conversational assistant that integrates multimodal retrieval and large language models (LLMs) to enable natural language interaction with heterogeneous agricultural and geospatial data. The proposed architecture combines orthophotos, Sentinel-2 vegetation indices, and user-provided documents through retrieval-augmented generation (RAG), allowing the system to flexibly determine whether to rely on multimodal evidence, textual knowledge, or both in formulating an answer. To assess response quality, we adopt an LLM-as-a-judge methodology using Qwen3-32B in a zero-shot, unsupervised setting, applying direct scoring in a multi-dimensional quantitative evaluation framework. Preliminary results show that the system is capable of generating clear, relevant, and context-aware responses to agricultural queries, while remaining reproducible and scalable across geographic regions. The primary contributions of this work include an architecture for fusing multimodal EO and textual knowledge sources, a demonstration of lowering the barrier to access specialized agricultural information through natural language interaction, and an open and reproducible design.