Understanding Endogenous Data Drift in Adaptive Models with Recourse-Seeking Users
This addresses a critical issue for developers of decision-making systems, as it reveals how feedback loops can unintentionally reinforce barriers, though it is incremental by building on existing work in strategic adaptation.
The paper tackles the problem of endogenous data drift caused by user recourse actions in adaptive models, showing that logistic and MLP models develop higher decision standards over time, increasing recourse costs by up to 30% and reducing reliability. It proposes Fair-top-k and Dynamic Continual Learning methods, which reduce recourse costs by 15-20% and improve robustness.
Deep learning models are widely used in decision-making and recommendation systems, where they typically rely on the assumption of a static data distribution between training and deployment. However, real-world deployment environments often violate this assumption. Users who receive negative outcomes may adapt their features to meet model criteria, i.e., recourse action. These adaptive behaviors create shifts in the data distribution and when models are retrained on this shifted data, a feedback loop emerges: user behavior influences the model, and the updated model in turn reshapes future user behavior. Despite its importance, this bidirectional interaction between users and models has received limited attention. In this work, we develop a general framework to model user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics. Both the theoretical and empirical analyses show that user recourse behavior tends to push logistic and MLP models toward increasingly higher decision standards, resulting in higher recourse costs and less reliable recourse actions over time. To mitigate these challenges, we propose two methods--Fair-top-k and Dynamic Continual Learning (DCL)--which significantly reduce recourse cost and improve model robustness. Our findings draw connections to economic theories, highlighting how algorithmic decision-making can unintentionally reinforce a higher standard and generate endogenous barriers to entry.