The Vacuity of the Open Source Security Testing Methodology Manual
This work highlights fundamental issues in a widely referenced security methodology, potentially impacting security testing professionals and standards.
The paper critiques the Open Source Security Testing Methodology Manual (OSSTMM), arguing it fails to deliver actual security by showing its unified security score is an empty abstraction, its trust metric is meaningless, and its approach to human security is flawed.
The Open Source Security Testing Methodology Manual (OSSTMM) provides a "scientific methodology for the accurate characterization of operational security" [Her10, p.13]. It is extensively referenced in writings aimed at security testing professionals such as textbooks, standards and academic papers. In this work we offer a fundamental critique of OSSTMM and argue that it fails to deliver on its promise of actual security. Our contribution is threefold and builds on a textual critique of this methodology. First, OSSTMM's central principle is that security can be understood as a quantity of which an entity has more or less. We show why this is wrong and how OSSTMM's unified security score, the rav, is an empty abstraction. Second, OSSTMM disregards risk by replacing it with a trust metric which confuses multiple definitions of trust and, as a result, produces a meaningless score. Finally, OSSTMM has been hailed for its attention to human security. Yet it understands all human agency as a security threat that needs to be constantly monitored and controlled. Thus, we argue that OSSTMM is neither fit for purpose nor can it be salvaged, and it should be abandoned by security professionals.