PASTRAMI: Performance Assessment of SofTware Routers Addressing Measurement Instability
This work addresses the need for reliable performance evaluation in virtualized networking environments, which is crucial for researchers and practitioners, but it is incremental as it builds on existing measurement challenges.
The paper tackles the problem of measurement variability in virtualized environments for assessing software router performance, specifically focusing on the Partial Drop Rate at 0.5% (PDR@0.5%), and proposes PASTRAMI, a methodology to differentiate stable from unstable configurations, revealing that some Linux versions exhibit non-negligible packet loss and high variability.
Virtualized environments offer a flexible and scalable platform for evaluating network performance, but they can introduce significant variability that complicates accurate measurement. This paper presents PASTRAMI, a methodology designed to assess the stability of software routers, which is critical to accurately evaluate performance metrics such as the Partial Drop Rate at 0.5% (PDR@0.5%). While PDR@0.5% is a key metric to assess packet processing capabilities of a software router, its reliable evaluation depends on consistent router performance with minimal measurement variability. Our research reveals that different Linux versions exhibit distinct behaviors, with some demonstrating non-negligible packet loss even at low loads and high variability in loss measurements, rendering them unsuitable for accurate performance assessments. This paper proposes a systematic approach to differentiate between stable and unstable environments, offering practical guidance on selecting suitable configurations for robust networking performance evaluations in virtualized environments.