QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments
This addresses the problem of establishing standardized testing and debugging tools for researchers in quantum software engineering, though it is incremental as it adapts existing benchmark concepts to a new domain.
The authors tackled the lack of reproducibility and comparability in quantum software engineering research by proposing QBugs, a benchmark that provides experimental subjects and infrastructure to facilitate evaluation and reproducibility of results.
Reproducibility and comparability of empirical results are at the core tenet of the scientific method in any scientific field. To ease reproducibility of empirical studies, several benchmarks in software engineering research, such as Defects4J, have been developed and widely used. For quantum software engineering research, however, no benchmark has been established yet. In this position paper, we propose a new benchmark -- named QBugs -- which will provide experimental subjects and an experimental infrastructure to ease the evaluation of new research and the reproducibility of previously published results on quantum software engineering.