Redbench: Workload Synthesis From Cloud Traces
For researchers and engineers benchmarking cloud data warehouses, Redbench provides a more realistic workload synthesis method that captures real-world patterns, addressing a known gap in existing benchmarks.
Redbench introduces a workload generator that reproduces real-world cloud data warehouse workload characteristics, such as query repetition and string-heavy queries, which are missing in standard benchmarks like TPC-H and TPC-DS. Evaluation shows it produces more realistic and reproducible workloads and reveals the impact of system optimizations across four commercial platforms.
Workload traces from cloud data warehouse providers reveal that standard benchmarks such as TPC-H and TPC-DS fail to capture key characteristics of real-world workloads, including query repetition and string-heavy queries. In this paper, we introduce Redbench, a novel benchmark featuring a workload generator that reproduces real-world workload characteristics derived from traces released by cloud providers. Redbench integrates multiple workload generation techniques to tailor workloads to specific objectives, transforming existing benchmarks into realistic query streams that preserve intrinsic workload characteristics. By focusing on inherent workload signals rather than execution-specific metrics, Redbench bridges the gap between synthetic and real workloads. Our evaluation shows that (1) Redbench produces more realistic and reproducible workloads for cloud data warehouse benchmarking, and (2) Redbench reveals the impact of system optimizations across four commercial data warehouse platforms. We believe that Redbench provides a crucial foundation for advancing research on optimization techniques for modern cloud data warehouses.