A Nonparametric Bayes Approach to Online Activity Prediction
This work addresses the need for accurate user activity prediction in online A/B testing, which is crucial for experimental planning, but it appears incremental as it builds on existing Bayesian nonparametric approaches.
The paper tackles the problem of predicting the number of users active in online experiments and the time needed to reach participation thresholds, proposing a Bayesian nonparametric model that outperforms existing methods in experiments on synthetic and real-world data.
Accurately predicting the onset of specific activities within defined timeframes holds significant importance in several applied contexts. In particular, accurate prediction of the number of future users that will be exposed to an intervention is an important piece of information for experimenters running online experiments (A/B tests). In this work, we propose a novel approach to predict the number of users that will be active in a given time period, as well as the temporal trajectory needed to attain a desired user participation threshold. We model user activity using a Bayesian nonparametric approach which allows us to capture the underlying heterogeneity in user engagement. We derive closed-form expressions for the number of new users expected in a given period, and a simple Monte Carlo algorithm targeting the posterior distribution of the number of days needed to attain a desired number of users; the latter is important for experimental planning. We illustrate the performance of our approach via several experiments on synthetic and real world data, in which we show that our novel method outperforms existing competitors.