A Study of "Churn" in Tweets and Real-Time Search Queries (Extended Version)
This addresses the challenge of computing term statistics for real-time search on Twitter, which is incremental as it builds on existing ranking functions.
The paper analyzed the rapid change in term distributions, termed 'churn', in tweets and search queries on Twitter, revealing insights into temporal dynamics and implications for search system design.
The real-time nature of Twitter means that term distributions in tweets and in search queries change rapidly: the most frequent terms in one hour may look very different from those in the next. Informally, we call this phenomenon "churn". Our interest in analyzing churn stems from the perspective of real-time search. Nearly all ranking functions, machine-learned or otherwise, depend on term statistics such as term frequency, document frequency, as well as query frequencies. In the real-time context, how do we compute these statistics, considering that the underlying distributions change rapidly? In this paper, we present an analysis of tweet and query churn on Twitter, as a first step to answering this question. Analyses reveal interesting insights on the temporal dynamics of term distributions on Twitter and hold implications for the design of search systems.