Cloud WorkBench - Infrastructure-as-Code Based Cloud Benchmarking
This addresses the problem for users of Infrastructure-as-a-Service clouds who need to evaluate costs and performance to optimize application deployment, representing an incremental improvement over existing benchmarking methods.
The paper tackles the cumbersome and error-prone process of benchmarking cloud services by proposing a cloud benchmarking Web service based on Infrastructure-as-Code, enabling reusable and representative benchmarks, as demonstrated in a case study measuring disk IO speeds of different Amazon EC2 instance and storage types.
To optimally deploy their applications, users of Infrastructure-as-a-Service clouds are required to evaluate the costs and performance of different combinations of cloud configurations to find out which combination provides the best service level for their specific application. Unfortunately, benchmarking cloud services is cumbersome and error-prone. In this paper, we propose an architecture and concrete implementation of a cloud benchmarking Web service, which fosters the definition of reusable and representative benchmarks. In distinction to existing work, our system is based on the notion of Infrastructure-as-Code, which is a state of the art concept to define IT infrastructure in a reproducible, well-defined, and testable way. We demonstrate our system based on an illustrative case study, in which we measure and compare the disk IO speeds of different instance and storage types in Amazon EC2.