What Makes a Scientific Paper be Accepted for Publication?
This work addresses the lack of transparency in peer-review processes for researchers and publishers, though it is incremental as it applies existing methods to a new domain.
The paper tackles the problem of understanding peer-review decisions by using machine learning and explainable AI to analyze linguistic features from an open peer-review dataset, finding that originality, clarity, and substance are key factors in acceptance recommendations for ICLR submissions.
Despite peer-reviewing being an essential component of academia since the 1600s, it has repeatedly received criticisms for lack of transparency and consistency. We posit that recent work in machine learning and explainable AI provide tools that enable insights into the decisions from a given peer review process. We start by extracting global explanations in the form of linguistic features that affect the acceptance of a scientific paper for publication on an open peer-review dataset. Second, since such global explanations do not justify causal interpretations, we provide a methodology for detecting confounding effects in natural language in order to generate causal explanations, under assumptions, in the form of lexicons. Our proposed linguistic explanation methodology indicates the following on a case dataset of ICLR submissions: a) the organising committee follows, for the most part, the recommendations of reviewers, and, b) the paper's main characteristics that led to reviewers recommending acceptance for publication are originality, clarity and substance.