How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction
For practitioners of churn prediction, this work highlights that observation-window sufficiency is design-dependent and not generalizable, cautioning against one-size-fits-all claims.
The paper investigates how many days of early user behavior are sufficient for subscription churn prediction, finding a diminishing-returns knee at 45-90 days in the KKBox dataset, but showing that window-sufficiency curves vary significantly with cohort/task design, achieving a +0.10 PR gain at 120 days for manual-renewal segments.
How many days of early behavior suffice for subscription churn prediction? In the public KKBox dataset, the early indicator of churn is typically an indicator of someone's contract status; however, when looking in the heavily churned manual-renewal segment, having access to early behavior creates a substantial increase in prediction for that specific segment (PR +0.10 at 120 days). A nine-window sufficiency curve shows a diminishing-returns knee in a 45-90 day band. However, stress-testing over three cohort/task designs shows that this curve is singular to the design being tested; for example, in our test with a moving target, the curve inverts and can shift depending on the feature set used. Therefore, any window-sufficiency claim should state its cohort construction, target definition, and feature families. All evidence is from one music-streaming dataset; the mechanism should generalize but the magnitudes may not.