Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
This addresses the threat of machine-generated fake reviews in online systems, which can deceive users and undermine trust, though it is incremental as it builds on prior methods to enhance context specificity.
The paper tackled the problem of generating context-specific fake restaurant reviews by developing a neural machine translation-based method that improves on-topic consistency, achieving near-optimal undetectability with a class-averaged F-score of 47% and significantly higher evasion rates (3.2/4 vs 1.5/4) compared to state-of-the-art.
Automatically generated fake restaurant reviews are a threat to online review systems. Recent research has shown that users have difficulties in detecting machine-generated fake reviews hiding among real restaurant reviews. The method used in this work (char-LSTM ) has one drawback: it has difficulties staying in context, i.e. when it generates a review for specific target entity, the resulting review may contain phrases that are unrelated to the target, thus increasing its detectability. In this work, we present and evaluate a more sophisticated technique based on neural machine translation (NMT) with which we can generate reviews that stay on-topic. We test multiple variants of our technique using native English speakers on Amazon Mechanical Turk. We demonstrate that reviews generated by the best variant have almost optimal undetectability (class-averaged F-score 47%). We conduct a user study with skeptical users and show that our method evades detection more frequently compared to the state-of-the-art (average evasion 3.2/4 vs 1.5/4) with statistical significance, at level α = 1% (Section 4.3). We develop very effective detection tools and reach average F-score of 97% in classifying these. Although fake reviews are very effective in fooling people, effective automatic detection is still feasible.