Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network
This work addresses the urgent need for spam detection systems to adapt to the LLM era, offering a practical solution for online platforms, though it is incremental as it builds on existing methods like embeddings and graph neural networks.
The paper tackles the problem of detecting spam reviews generated by large language models (LLMs), which mimic human writing and threaten online platforms, by proposing FraudSquad, a hybrid model that integrates language model embeddings and a graph neural network, achieving up to 44.22% higher precision and 43.01% higher recall than state-of-the-art baselines on LLM-generated datasets.
The rise of large language models (LLMs) has enabled the generation of highly persuasive spam reviews that closely mimic human writing. These reviews pose significant challenges for existing detection systems and threaten the credibility of online platforms. In this work, we first create three realistic LLM-generated spam review datasets using three distinct LLMs, each guided by product metadata and genuine reference reviews. Evaluations by GPT-4.1 confirm the high persuasion and deceptive potential of these reviews. To address this threat, we propose FraudSquad, a hybrid detection model that integrates text embeddings from a pre-trained language model with a gated graph transformer for spam node classification. FraudSquad captures both semantic and behavioral signals without relying on manual feature engineering or massive training resources. Experiments show that FraudSquad outperforms state-of-the-art baselines by up to 44.22% in precision and 43.01% in recall on three LLM-generated datasets, while also achieving promising results on two human-written spam datasets. Furthermore, FraudSquad maintains a modest model size and requires minimal labeled training data, making it a practical solution for real-world applications. Our contributions include new synthetic datasets, a practical detection framework, and empirical evidence highlighting the urgency of adapting spam detection to the LLM era. Our code and datasets are available at: https://anonymous.4open.science/r/FraudSquad-5389/.