Differentiating Between Human-Written and AI-Generated Texts Using Automatically Extracted Linguistic Features
This incremental work addresses the problem of distinguishing AI-generated text for researchers and developers, highlighting gaps in AI's ability to emulate human writing.
The study compared linguistic features between human-written and AI-generated essays, finding significant differences in consonants, nouns, adjectives, pronouns, modifiers, and difficult words, despite AI's mimicry of human speech.
While extensive research has focused on ChatGPT in recent years, very few studies have systematically quantified and compared linguistic features between human-written and artificial intelligence (AI)-generated language. This exploratory study aims to investigate how various linguistic components are represented in both types of texts, assessing the ability of AI to emulate human writing. Using human-authored essays as a benchmark, we prompted ChatGPT to generate essays of equivalent length. These texts were analyzed using Open Brain AI, an online computational tool, to extract measures of phonological, morphological, syntactic, and lexical constituents. Despite AI-generated texts appearing to mimic human speech, the results revealed significant differences across multiple linguistic features such as specific types of consonants, nouns, adjectives, pronouns, adjectival/prepositional modifiers, and use of difficult words, among others. These findings underscore the importance of integrating automated tools for efficient language assessment, reducing time and effort in data analysis. Moreover, they emphasize the necessity for enhanced training methodologies to improve the engineering capacity of AI for producing more human-like text.