An Independent Validation of Vulnerability Discovery Models
This work provides incremental validation for software security assessment by evaluating existing models on new browser data, helping practitioners choose more reliable VDMs.
The authors conducted an independent validation of six vulnerability discovery models (VDMs) across seventeen releases of three popular browsers, finding that data sets based on 'confirmed-by-vendors' advisories yield more stable and better results, with the s-shape logistic model (AML) performing best overall, while the Anderson thermodynamic model (AT) was unsuitable.
Having a precise vulnerability discovery model (VDM) would provide a useful quantitative insight to assess software security. Thus far, several models have been proposed with some evidence supporting their goodness-of-fit. In this work we describe an independent validation of the applicability of six existing VDMs in seventeen releases of the three popular browsers Firefox, Google Chrome and Internet Explorer. We have collected five different kinds of data sets based on different definitions of a vulnerability. We introduce two quantitative metrics, goodness-of-fit entropy and goodness-of-fit quality, to analyze the impact of vulnerability data sets to the stability as well as quality of VDMs in the software life cycles. The experiment result shows that the "confirmed-by-vendors' advisories" data sets apparently yields more stable and better results for VDMs. And the performance of the s-shape logistic model (AML) seems to be superior performance in overall. Meanwhile, Anderson thermodynamic model (AT) is indeed not suitable for modeling the vulnerability discovery process. This means that the discovery process of vulnerabilities and normal bugs are different because the interests of people in finding security vulnerabilities are more than finding normal programming bugs.