A Systematically Empirical Evaluation of Vulnerability Discovery Models: a Study on Browsers' Vulnerabilities
This work addresses the need for systematic evaluation of VDMs to help software vendors and users assess security trends, though it is incremental as it builds on existing models.
The authors tackled the lack of independent validation for vulnerability discovery models (VDMs) by introducing an empirical methodology to evaluate their quality and predictability, finding that some models should be rejected while simpler linear models work best for young browsers and logistics-based models for others.
A precise vulnerability discovery model (VDM) will provide a useful insight to assess software security, and could be a good prediction instrument for both software vendors and users to understand security trends and plan ahead patching schedule accordingly. Thus far, several models have been proposed and validated. Yet, no systematically independent validation by somebody other than the author exists. Furthermore, there are a number of issues that might bias previous studies in the field. In this work, we fill in the gap by introducing an empirical methodology that systematically evaluates the performance of a VDM in two aspects: quality and predictability. We further apply this methodology to assess existing VDMs. The results show that some models should be rejected outright, while some others might be adequate to capture the discovery process of vulnerabilities. We also consider different usage scenarios of VDMs and find that the simplest linear model is the most appropriate choice in terms of both quality and predictability when browsers are young. Otherwise, logistics-based models are better choices.