Asking Grok: AI-Assisted Sensemaking in Social Media Conversations
For social media platforms and researchers, this work provides the first large-scale empirical analysis of real-world user interactions with an LLM-powered AI assistant in social media conversations, revealing its limited and complementary role to existing fact-checking systems.
This study analyzes 169,137 posts invoking Grok on X, finding that the AI assistant is primarily used reactively for information verification, reaches small audiences, and is used once by 76.8% of users. Grok interactions occur early and do not predict later Community Notes corrections, suggesting AI assistants serve as an early complementary sensemaking layer rather than a replacement for crowd-based fact-checking.
LLM-powered AI assistants (e.g., Grok) are increasingly integrated into social media platforms, where they help explain content, provide context, and verify claims directly within conversation threads. While prior research has examined the accuracy of LLMs for fact-checking, little is known about how people interact with such systems in real-world social media environments. In this study, we empirically analyze user interactions with the AI assistant Grok on the social media platform X. Using a large-scale dataset consisting of 169,137 posts invoking Grok, we examine the types of requests directed at the AI assistant and the contexts in which it is used. We find that Grok is primarily invoked reactively to obtain or verify information. Although responses appear quickly, they typically only reach small audiences. Adoption is widespread but shallow, with 76.8% of users invoking Grok just once. We further examine how these interactions relate to Community Notes, X's community-based fact-checking system. While overlap between both systems is limited, it concentrates on verification-oriented and high-visibility content. Grok interactions typically occur earlier and do not predict subsequent correction activity. Together, these findings suggest that AI assistants function as an early complementary layer of sensemaking on social media rather than a replacement for crowd-based fact-checking systems.