Feedback Detection for Live Predictors
This addresses a critical issue for machine learning practitioners deploying models in production systems, though it appears incremental as it builds on causal inference methods.
The paper tackles the problem of feedback loops in live predictors, where models may cause the behaviors they predict, and introduces a local randomization scheme for detecting non-linear feedback, demonstrated in a pilot study with a deployed search engine system.
A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze predictor feedback detection as a causal inference problem, and introduce a local randomization scheme that can be used to detect non-linear feedback in real-world problems. We conduct a pilot study for our proposed methodology using a predictive system currently deployed as a part of a search engine.