On the estimation of discrete choice models to capture irrational customer behaviors
This work addresses the challenge of accurately modeling irrational consumer choices in retail, which is incremental as it builds on existing generalized models but improves estimation to reduce overfitting risks.
The authors tackled the problem of estimating discrete choice models that can capture irrational customer behaviors, such as halo effects, which are not handled by traditional Random Utility Maximization models, and their proposed method for the Generalized Stochastic Preference model achieved a 12.5% average boost in predictive accuracy on a real-world grocery and drug store dataset.
The Random Utility Maximization model is by far the most adopted framework to estimate consumer choice behavior. However, behavioral economics has provided strong empirical evidence of irrational choice behavior, such as halo effects, that are incompatible with this framework. Models belonging to the Random Utility Maximization family may therefore not accurately capture such irrational behavior. Hence, more general choice models, overcoming such limitations, have been proposed. However, the flexibility of such models comes at the price of increased risk of overfitting. As such, estimating such models remains a challenge. In this work, we propose an estimation method for the recently proposed Generalized Stochastic Preference choice model, which subsumes the family of Random Utility Maximization models and is capable of capturing halo effects. Specifically, we show how to use partially-ranked preferences to efficiently model rational and irrational customer types from transaction data. Our estimation procedure is based on column generation, where relevant customer types are efficiently extracted by expanding a tree-like data structure containing the customer behaviors. Further, we propose a new dominance rule among customer types whose effect is to prioritize low orders of interactions among products. An extensive set of experiments assesses the predictive accuracy of the proposed approach. Our results show that accounting for irrational preferences can boost predictive accuracy by 12.5% on average, when tested on a real-world dataset from a large chain of grocery and drug stores.