White-Box Analysis over Machine Learning: Modeling Performance of Configurable Systems
This addresses the trade-offs in modeling configurable system performance for stakeholders, offering a more efficient and interpretable alternative to black-box techniques, though it is incremental as it builds on existing modeling concepts.
The paper tackles the problem of building performance-influence models for configurable systems by introducing Comprex, a white-box approach that avoids machine learning extrapolation. It demonstrates that Comprex achieves similar accuracy to the most accurate black-box method but at reduced cost, with benefits like interpretability and local models, as shown in evaluations on 4 open-source projects.
Performance-influence models can help stakeholders understand how and where configuration options and their interactions influence the performance of a system. With this understanding, stakeholders can debug performance behavior and make deliberate configuration decisions. Current black-box techniques to build such models combine various sampling and learning strategies, resulting in tradeoffs between measurement effort, accuracy, and interpretability. We present Comprex, a white-box approach to build performance-influence models for configurable systems, combining insights of local measurements, dynamic taint analysis to track options in the implementation, compositionality, and compression of the configuration space, without relying on machine learning to extrapolate incomplete samples. Our evaluation on 4 widely-used, open-source projects demonstrates that Comprex builds similarly accurate performance-influence models to the most accurate and expensive black-box approach, but at a reduced cost and with additional benefits from interpretable and local models.