Which stylistic features fool ChatGPT research evaluations?
This identifies a potential bias in using LLMs for research evaluation, which could affect researchers and institutions relying on automated assessments.
The study investigated whether linguistic features unrelated to research quality influence ChatGPT's evaluation scores of journal articles, finding that linguistic complexity and abstract length were more strongly correlated with ChatGPT scores than with expert REF scores across many fields.
Large Language Models (LLMs) have the potential to be used to support research evaluation and have a moderate capability to estimate the research quality of a journal article from its title and abstract. This paper assesses whether there are language-related factors unrelated to the quality of the research that influence ChatGPT's scores. Using a dataset of 99,277 journal articles submitted to the UK-wide Research Excellence Framework (REF) 2021 assessments, we calculated several readability indicators from abstracts and correlated them with ChatGPT scores and departmental REF scores. From the results, linguistic complexity and length were more strongly associated with ChatGPT research quality scores than with REF expert scores in many subject areas. Although cause-and-effect was not tested, these results suggest that ChatGPT may be more likely than human experts to reward linguistic complexity, with a potential bias towards longer and less readable abstracts in many fields. The apparent preference of LLMs for complex language is an undesirable feature for practical applications of LLMs for research quality evaluation, unless solutions can be found.