repro_eval: A Python Interface to Reproducibility Measures of System-oriented IR Experiments
This tool addresses the need for standardized reproducibility assessment in information retrieval research, though it is incremental as it builds on existing concepts without introducing new methods.
The authors introduced repro_eval, a Python tool for measuring reproducibility in system-oriented information retrieval experiments, providing researchers with extensible measures to evaluate system outputs and promote common practices in reproducibility studies.
In this work we introduce repro_eval - a tool for reactive reproducibility studies of system-oriented information retrieval (IR) experiments. The corresponding Python package provides IR researchers with measures for different levels of reproduction when evaluating their systems' outputs. By offering an easily extensible interface, we hope to stimulate common practices when conducting a reproducibility study of system-oriented IR experiments.