ARENA: An Architecture for Measuring the Transferability of Autonomous Cyber Defense
For researchers in autonomous cyber defense, ARENA provides a way to use realistic production data without releasing raw logs, addressing the reproducibility crisis in SOC research.
The paper presents ARENA, a methodology for extracting, anonymizing, structuring, and validating SIEM data from a production financial SOC while preserving task-relevant structure. It demonstrates that the resulting artifacts enable reproducible cyber defense research, with 37 MITRE ATT&CK-mapped challenges and a verifier that detects non-compliant LLM actions across 200 incidents.
Operational evidence is not automatically scientific evidence. The most realistic Security Operations Center (SOC) data is production telemetry, yet it remains scientifically inaccessible because raw logs cannot be released; as a result, research relies on synthetic or dated datasets. We treat the boundary between private production telemetry and reusable research artifacts as the design object: a methodology that extracts, anonymizes, structures, and validates Security Information and Event Management (SIEM) data from a production financial SOC while preserving task-relevant investigative structure within a declared privacy boundary. Two consumers stress the same artifact. As training material, it fails loudly: 37 MITRE ATT&CK-mapped HIKARI challenges work only when anonymization preserves temporal order and entity consistency. As a measurement substrate, it fails quietly: across 200 SOCpilot incidents, a deterministic verifier detects non-compliant Large Language Model (LLM) actions that are absent from the human baseline. The result is a measurable privacy-utility boundary rather than a formal anonymity claim.