StorRep: Storage Research Experiment Patterns on Chameleon Cloud and Trovi
For storage researchers, this work addresses the reproducibility crisis by providing patterns and guidelines for creating transparent and dependable experiments.
Only 1% of SSD simulator-based experiments are packaged and 0.5% are easily reproducible; StorRep provides six extensible and reproducible storage experiment artifacts on Chameleon Cloud to address this.
Storage experiments are vital to advancing storage research, but creating extensible and reproducible storage artifacts can be a challenging task. Our research has shown that only 1% of SSD simulator-based experiences are packaged and 0.5% of them can be easily reproduced. The lack of such artifacts without proper reproducibility can significantly impede the advancement of storage research. The biggest challenges in these types of experiments are ensuring that we have the correct environment to conduct them and creating extensible experiments that can be built upon in future research. To address this issue, we introduce StorRep, a thorough study that provides six extensible and reproducible storage experiment artifacts that serve as the foundation for further storage research, utilizing the Chameleon infrastructure. Our study offers experiment patterns and guidelines that can help researchers create transparent and dependable storage experiments. We have successfully integrated our methods in several experiments in multiple community and educational events over several years and produced publicly accessible artifacts that can be extended and fully reproduced without any restrictions.