Developing Security Reputation Metrics for Hosting Providers
This work addresses the need for security reputation metrics in cybercrime prevention, though it is incremental as it builds on existing abuse data approaches.
The paper tackled the lack of systematic security metrics for hosting providers by developing a method to identify and rank providers based on abuse data, addressing challenges like normalization and aggregation, and applied it to answer a specific question from the Dutch police about the worst providers in their jurisdiction.
Research into cybercrime often points to concentrations of abuse at certain hosting providers. The implication is that these providers are worse in terms of security; some are considered `bad' or even `bullet proof'. Remarkably little work exists on systematically comparing the security performance of providers. Existing metrics typically count instances of abuse and sometimes normalize these counts by taking into account the advertised address space of the provider. None of these attempts have worked through the serious methodological challenges that plague metric design. In this paper we present a systematic approach for metrics development and identify the main challenges: (i) identification of providers, (ii) abuse data coverage and quality, (iii) normalization, (iv) aggregation and (v) metric interpretation. We describe a pragmatic approach to deal with these challenges. In the process, we answer an urgent question posed to us by the Dutch police: `which are the worst providers in our jurisdiction?'. Notwithstanding their limitations, there is a clear need for security metrics for hosting providers in the fight against cybercrime.