Performative Prediction with Neural Networks
This work addresses the challenge of performative prediction for machine learning systems that influence their own data, offering a more practical framework for real-world applications.
The paper tackles the problem of learning performatively stable classifiers under more realistic assumptions, specifically by assuming Lipschitz continuity with respect to predictions rather than model parameters, which allows relaxing loss function requirements like convexity. They demonstrate this by learning stable classifiers with neural networks on real data using a proposed resampling procedure.
Performative prediction is a framework for learning models that influence the data they intend to predict. We focus on finding classifiers that are performatively stable, i.e. optimal for the data distribution they induce. Standard convergence results for finding a performatively stable classifier with the method of repeated risk minimization assume that the data distribution is Lipschitz continuous to the model's parameters. Under this assumption, the loss must be strongly convex and smooth in these parameters; otherwise, the method will diverge for some problems. In this work, we instead assume that the data distribution is Lipschitz continuous with respect to the model's predictions, a more natural assumption for performative systems. As a result, we are able to significantly relax the assumptions on the loss function. In particular, we do not need to assume convexity with respect to the model's parameters. As an illustration, we introduce a resampling procedure that models realistic distribution shifts and show that it satisfies our assumptions. We support our theory by showing that one can learn performatively stable classifiers with neural networks making predictions about real data that shift according to our proposed procedure.