Strategic Classification in the Dark
This addresses a problem in high-stake classification scenarios like credit scoring, where agents may game unknown classifiers, and is incremental as it generalizes an existing model.
The paper tackles strategic classification when agents do not know the classifier, defining the 'price of opacity' as the difference in prediction error between opaque and transparent robust classifiers, and shows through experiments that Hardt et al.'s robust classifier is affected by opacity, with transparency sometimes recommended.
Strategic classification studies the interaction between a classification rule and the strategic agents it governs. Under the assumption that the classifier is known, rational agents respond to it by manipulating their features. However, in many real-life scenarios of high-stake classification (e.g., credit scoring), the classifier is not revealed to the agents, which leads agents to attempt to learn the classifier and game it too. In this paper we generalize the strategic classification model to such scenarios. We define the price of opacity as the difference in prediction error between opaque and transparent strategy-robust classifiers, characterize it, and give a sufficient condition for this price to be strictly positive, in which case transparency is the recommended policy. Our experiments show how Hardt et al.'s robust classifier is affected by keeping agents in the dark.