Analysis of hidden feedback loops in continuous machine learning systems
This addresses a foundational problem in deploying reliable AI systems, though it is a conceptual analysis without experimental validation.
The paper identifies the problem of hidden feedback loops in continuous machine learning systems, where AI interactions cause concept drift, and demonstrates this issue using a housing prices prediction system example.
In this concept paper, we discuss intricacies of specifying and verifying the quality of continuous and lifelong learning artificial intelligence systems as they interact with and influence their environment causing a so-called concept drift. We signify a problem of implicit feedback loops, demonstrate how they intervene with user behavior on an exemplary housing prices prediction system. Based on a preliminary model, we highlight conditions when such feedback loops arise and discuss possible solution approaches.