User Prompting Strategies and ChatGPT Contextual Adaptation Shape Conversational Information-Seeking Experiences
For researchers and designers of conversational AI, this work reveals behavioral disparities in user prompting and contextual adaptation, highlighting the digital divide and the nuanced effects of AI response styles on user attitudes.
This study analyzed real-world conversational information-seeking with ChatGPT in a nationally representative sample of 937 U.S. adults, finding that only 19.1% of users employed prompting strategies (disproportionately more educated and Democrat-leaning) and that ChatGPT adapts its responses contextually, with controversial topics eliciting more cognitively complex responses that were perceived less favorably but produced more positive issue-relevant attitudes.
Conversational AI, such as ChatGPT, is increasingly used for information seeking. However, little is known about how ordinary users actually prompt and how ChatGPT adapts its responses in real-world conversational information seeking (CIS). In this study, a nationally representative sample of 937 U.S. adults engaged in multi-turn CIS with ChatGPT on both controversial and non-controversial topics across science, health, and policy contexts. We analyzed both user prompting strategies and the communication styles of ChatGPT responses. The findings revealed behavioral signals of digital divide: only 19.1% of users employed prompting strategies, and these users were disproportionately more educated and Democrat-leaning. Further, ChatGPT demonstrated contextual adaptation: responses to controversial topics contain more cognitive complexity and more external references than to non-controversial topics. Notably, cognitively complex responses were perceived as less favorable but produced more positive issue-relevant attitudes. This study highlights disparities in user prompting behaviors and shows how user prompts and AI responses together shape information-seeking with conversational AI.