Evaluating Large Language Models for IUCN Red List Species Information
It addresses the reliability of LLMs for conservation decision-making, highlighting risks like biases and the need for human oversight, making it an incremental but critical domain-specific contribution.
This study evaluated five large language models on 21,955 species for IUCN Red List assessments, finding they excelled at taxonomic classification (94.9%) but consistently failed at conservation reasoning (27.2% for status assessment), revealing a knowledge-reasoning gap.
Large Language Models (LLMs) are rapidly being adopted in conservation to address the biodiversity crisis, yet their reliability for species evaluation is uncertain. This study systematically validates five leading models on 21,955 species across four core IUCN Red List assessment components: taxonomy, conservation status, distribution, and threats. A critical paradox was revealed: models excelled at taxonomic classification (94.9%) but consistently failed at conservation reasoning (27.2% for status assessment). This knowledge-reasoning gap, evident across all models, suggests inherent architectural constraints, not just data limitations. Furthermore, models exhibited systematic biases favoring charismatic vertebrates, potentially amplifying existing conservation inequities. These findings delineate clear boundaries for responsible LLM deployment: they are powerful tools for information retrieval but require human oversight for judgment-based decisions. A hybrid approach is recommended, where LLMs augment expert capacity while human experts retain sole authority over risk assessment and policy.