Ecological Evaluation of Persuasive Messages Using Google AdWords
This provides a practical solution for researchers in NLP and persuasion studies to conduct experiments more efficiently, though it is incremental in leveraging existing tools.
The paper tackles the problem of evaluating persuasive messages in NLP by introducing a cheap, fast methodology using Google AdWords to measure message impact in ecological settings, with pilot experiments confirming its effectiveness.
In recent years there has been a growing interest in crowdsourcing methodologies to be used in experimental research for NLP tasks. In particular, evaluation of systems and theories about persuasion is difficult to accommodate within existing frameworks. In this paper we present a new cheap and fast methodology that allows fast experiment building and evaluation with fully-automated analysis at a low cost. The central idea is exploiting existing commercial tools for advertising on the web, such as Google AdWords, to measure message impact in an ecological setting. The paper includes a description of the approach, tips for how to use AdWords for scientific research, and results of pilot experiments on the impact of affective text variations which confirm the effectiveness of the approach.