AGRI-Fidelity: Evaluating the Reliability of Listenable Explanations for Poultry Disease Detection
This addresses the issue of unreliable explanations in noisy farm environments for poultry disease detection, representing an incremental improvement in evaluation methods for XAI.
The paper tackles the problem of evaluating the reliability of listenable explanations in poultry disease detection, where existing metrics fail to account for model multiplicity and stationary artifacts like ventilation noise, and proposes AGRI-Fidelity, a framework that uses cross-model consensus and cyclic temporal permutation to compute a False Discovery Rate, effectively providing reliability-aware discrimination across datasets.
Existing XAI metrics measure faithfulness for a single model, ignoring model multiplicity where near-optimal classifiers rely on different or spurious acoustic cues. In noisy farm environments, stationary artifacts such as ventilation noise can produce explanations that are faithful yet unreliable, as masking-based metrics fail to penalize redundant shortcuts. We propose AGRI-Fidelity, a reliability-oriented evaluation framework for listenable explanations in poultry disease detection without spatial ground truth. The method combines cross-model consensus with cyclic temporal permutation to construct null distributions and compute a False Discovery Rate (FDR), suppressing stationary artifacts while preserving time-localized bioacoustic markers. Across real and controlled datasets, AGRI-Fidelity effectively provides reliability-aware discrimination for all data points versus masking-based metrics.