AVION: Aerial Vision-Language Instruction from Offline Teacher to Prompt-Tuned Network
This work addresses the problem of limited semantic coverage and adaptability in aerial vision-language models for remote sensing applications, representing an incremental advancement.
The paper tackles the challenge of adapting vision-language models to remote sensing imagery by proposing AVION, a knowledge distillation framework that improves few-shot classification and cross-modal retrieval on six benchmarks, achieving enhanced accuracy and recall with minimal added parameters.
Adapting vision-language models to remote sensing imagery remains challenging due to two key factors: limited semantic coverage in textual representations and insufficient adaptability of visual features. These issues are particularly significant in aerial scenes, which involve various visual appearances and fine-grained object distinctions. We propose AVION, a knowledge distillation framework tailored for remote sensing adaptation of vision-language models. The teacher module constructs semantically rich textual prototypes by collecting descriptions from a large language model and verifying validity using remote sensing image features. The student module integrates lightweight and learnable prompts into both vision and language encoders, guided by the teacher to align embeddings and their cross-modal relationships. Once trained, the student operates independently during inference. Experiments on six optical remote sensing benchmarks show that AVION improves few-shot classification and base-class accuracy without degrading generalization to novel categories. It also enhances mean recall for cross-modal retrieval, with minimal additional trainable parameters.