A Semi-automated Peer-review System
This addresses bias and efficiency issues in peer review for scientific publishing, but it is incremental as it builds on existing methods without major breakthroughs.
The paper tackles bias and incompleteness in traditional peer review by proposing a semi-supervised, human-assisted classifier, evaluating it through hypothetical ROC curves to show potential benefits in automation for manuscript evaluation.
A semi-supervised model of peer review is introduced that is intended to overcome the bias and incompleteness of traditional peer review. Traditional approaches are reliant on human biases, while consensus decision-making is constrained by sparse information. Here, the architecture for one potential improvement (a semi-supervised, human-assisted classifier) to the traditional approach will be introduced and evaluated. To evaluate the potential advantages of such a system, hypothetical receiver operating characteristic (ROC) curves for both approaches will be assessed. This will provide more specific indications of how automation would be beneficial in the manuscript evaluation process. In conclusion, the implications for such a system on measurements of scientific impact and improving the quality of open submission repositories will be discussed.