Vulnerability Coverage as an Adequacy Testing Criterion
This addresses the challenge of ensuring software security against known vulnerabilities for developers and testers, though it is incremental as it builds on existing evolutionary methods.
The paper tackles the problem of exhaustive security testing for configurable software applications with many parameters by introducing 'vulnerability coverage' as an adequacy criterion, using evolutionary algorithms to generate test inputs based on CVSS scores and reporting their performance in identifying vulnerabilities from the NVD.
Mainstream software applications and tools are the configurable platforms with an enormous number of parameters along with their values. Certain settings and possible interactions between these parameters may harden (or soften) the security and robustness of these applications against some known vulnerabilities. However, the large number of vulnerabilities reported and associated with these tools make the exhaustive testing of these tools infeasible against these vulnerabilities infeasible. As an instance of general software testing problem, the research question to address is whether the system under test is robust and secure against these vulnerabilities. This paper introduces the idea of ``vulnerability coverage,'' a concept to adequately test a given application for a certain classes of vulnerabilities, as reported by the National Vulnerability Database (NVD). The deriving idea is to utilize the Common Vulnerability Scoring System (CVSS) as a means to measure the fitness of test inputs generated by evolutionary algorithms and then through pattern matching identify vulnerabilities that match the generated vulnerability vectors and then test the system under test for those identified vulnerabilities. We report the performance of two evolutionary algorithms (i.e., Genetic Algorithms and Particle Swarm Optimization) in generating the vulnerability pattern vectors.