Crowdsourcing the State of the Art(ifacts)
This addresses the challenge for researchers and educators in determining leading-edge research, though it is incremental as it builds on existing community monitoring methods.
The paper tackles the problem of identifying the state-of-the-art in research by proposing a crowdsourced reuse graph method, finding over 1,600 instances of reuse in a study of 170 software engineering papers from 2020, indicating rampant reuse.
In any field, finding the "leading edge" of research is an on-going challenge. Researchers cannot appease reviewers and educators cannot teach to the leading edge of their field if no one agrees on what is the state-of-the-art. Using a novel crowdsourced "reuse graph" approach, we propose here a new method to learn this state-of-the-art. Our reuse graphs are less effort to build and verify than other community monitoring methods (e.g. artifact tracks or citation-based searches). Based on a study of 170 papers from software engineering (SE) conferences in 2020, we have found over 1,600 instances of reuse; i.e., reuse is rampant in SE research. Prior pessimism about a lack of reuse in SE research may have been a result of using the wrong methods to measure the wrong things.