AI Revolution on Chat Bot: Evidence from a Randomized Controlled Experiment
This addresses the need for real-world evidence on AI tools for support services, but it appears incremental as it extends existing lab studies to field settings.
The paper tackles the problem of limited field experiments on LLM-based tools in realistic settings by conducting a randomized controlled trial to assess their effectiveness in providing unmonitored support services for information retrieval, but no concrete results or numbers are provided in the abstract.
In recent years, generative AI has undergone major advancements, demonstrating significant promise in augmenting human productivity. Notably, large language models (LLM), with ChatGPT-4 as an example, have drawn considerable attention. Numerous articles have examined the impact of LLM-based tools on human productivity in lab settings and designed tasks or in observational studies. Despite recent advances, field experiments applying LLM-based tools in realistic settings are limited. This paper presents the findings of a field randomized controlled trial assessing the effectiveness of LLM-based tools in providing unmonitored support services for information retrieval.