ChatGPT-generated texts show authorship traits that identify them as non-human
This work addresses the problem of detecting AI-generated text for applications in authenticity verification, though it is incremental in identifying specific linguistic differences.
The study investigated whether ChatGPT-generated texts have distinct authorship traits that differentiate them from human writing, finding that the model shows more limited stylistic variation across registers and prefers nouns over verbs, unlike humans who anchor language in tense, aspect, and mood.
Large Language Models can emulate different writing styles, ranging from composing poetry that appears indistinguishable from that of famous poets to using slang that can convince people that they are chatting with a human online. While differences in style may not always be visible to the untrained eye, we can generally distinguish the writing of different people, like a linguistic fingerprint. This work examines whether a language model can also be linked to a specific fingerprint. Through stylometric and multidimensional register analyses, we compare human-authored and model-authored texts from different registers. We find that the model can successfully adapt its style depending on whether it is prompted to produce a Wikipedia entry vs. a college essay, but not in a way that makes it indistinguishable from humans. Concretely, the model shows more limited variation when producing outputs in different registers. Our results suggest that the model prefers nouns to verbs, thus showing a distinct linguistic backbone from humans, who tend to anchor language in the highly grammaticalized dimensions of tense, aspect, and mood. It is possible that the more complex domains of grammar reflect a mode of thought unique to humans, thus acting as a litmus test for Artificial Intelligence.