Artificial Effort
For experimental economists, the paper shows that real-effort tasks may no longer measure genuine human effort when participants can outsource to LLMs, threatening the validity of a widely used experimental method.
The paper tests whether 8 canonical real-effort tasks from experimental economics can be automated by 23 LLMs from three providers, finding that most tasks are solved accurately at negligible cost, with performance improving across model generations and mid-tier models catching up to frontier ones. Monetary incentives do not affect LLM performance, establishing a boundary condition for using these tasks in unsupervised settings.
Real-effort tasks, in which participants perform cognitively costly activities whose outcomes depend on actual performance, are widely used in experimental economics. Their validity, however, rests on the assumption that a human performs them. We study whether this assumption still holds in the era of Artificial Intelligence (AI) and Large Language Models (LLMs). Using 8 canonical real-effort tasks and 23 LLMs from three major providers, we show that most tasks can now be solved accurately and at a negligible cost, while only a few resist automation. Performance improves with each model generation, and midtier models are rapidly closing the gap with frontier ones, broadening the set of widely accessible models that can automate these tasks. Additionally, we show that verbally offering monetary incentives has no effect on LLM performance. Our findings establish a boundary condition for the use of real-effort tasks in unsupervised settings: when participants can cheaply outsource task completion to an LLM, observed performance may no longer reflect genuine human effort.