The Achilles Heel of AI: Fundamentals of Risk-Aware Training Data for High-Consequence Models
This addresses the need for more robust and efficient AI development in critical domains like defense and disaster response, though it is incremental as it builds on existing annotation methods.
The paper tackles the problem of inefficient training data annotation for high-consequence AI models by introducing smart-sizing, a strategy that prioritizes label diversity and utility, resulting in models trained on 20-40% of curated data matching or exceeding full-data baselines in rare-class recall and edge-case generalization.
AI systems in high-consequence domains such as defense, intelligence, and disaster response must detect rare, high-impact events while operating under tight resource constraints. Traditional annotation strategies that prioritize label volume over informational value introduce redundancy and noise, limiting model generalization. This paper introduces smart-sizing, a training data strategy that emphasizes label diversity, model-guided selection, and marginal utility-based stopping. We implement this through Adaptive Label Optimization (ALO), combining pre-labeling triage, annotator disagreement analysis, and iterative feedback to prioritize labels that meaningfully improve model performance. Experiments show that models trained on 20 to 40 percent of curated data can match or exceed full-data baselines, particularly in rare-class recall and edge-case generalization. We also demonstrate how latent labeling errors embedded in training and validation sets can distort evaluation, underscoring the need for embedded audit tools and performance-aware governance. Smart-sizing reframes annotation as a feedback-driven process aligned with mission outcomes, enabling more robust models with fewer labels and supporting efficient AI development pipelines for frontier models and operational systems.