Automatic Image-Level Morphological Trait Annotation for Organismal Images
This work addresses the bottleneck of high-quality trait annotation for large-scale ecological studies by providing a scalable alternative to manual efforts, though it is incremental as it builds on existing foundation models and methods.
The paper tackled the problem of slow, expert-driven extraction of morphological traits from biological images by developing an automated pipeline that uses sparse autoencoders and vision-language prompting to generate trait annotations, resulting in Bioscan-Traits, a dataset of 80K annotations across 19K insect images with human-evaluated biological plausibility.
Morphological traits are physical characteristics of biological organisms that provide vital clues on how organisms interact with their environment. Yet extracting these traits remains a slow, expert-driven process, limiting their use in large-scale ecological studies. A major bottleneck is the absence of high-quality datasets linking biological images to trait-level annotations. In this work, we demonstrate that sparse autoencoders trained on foundation-model features yield monosemantic, spatially grounded neurons that consistently activate on meaningful morphological parts. Leveraging this property, we introduce a trait annotation pipeline that localizes salient regions and uses vision-language prompting to generate interpretable trait descriptions. Using this approach, we construct Bioscan-Traits, a dataset of 80K trait annotations spanning 19K insect images from BIOSCAN-5M. Human evaluation confirms the biological plausibility of the generated morphological descriptions. We assess design sensitivity through a comprehensive ablation study, systematically varying key design choices and measuring their impact on the quality of the resulting trait descriptions. By annotating traits with a modular pipeline rather than prohibitively expensive manual efforts, we offer a scalable way to inject biologically meaningful supervision into foundation models, enable large-scale morphological analyses, and bridge the gap between ecological relevance and machine-learning practicality.