Using Knowledge Distillation to improve interpretable models in a retail banking context
This work addresses the need for better predictive models in retail banking while maintaining interpretability for regulatory compliance, though it is incremental as it applies existing distillation methods to this domain.
The paper tackles the problem of improving simple, interpretable models in retail banking under regulatory constraints by applying knowledge distillation techniques, demonstrating their potential to enhance model performance without increasing complexity.
This article sets forth a review of knowledge distillation techniques with a focus on their applicability to retail banking contexts. Predictive machine learning algorithms used in banking environments, especially in risk and control functions, are generally subject to regulatory and technical constraints limiting their complexity. Knowledge distillation gives the opportunity to improve the performances of simple models without burdening their application, using the results of other - generally more complex and better-performing - models. Parsing recent advances in this field, we highlight three main approaches: Soft Targets, Sample Selection and Data Augmentation. We assess the relevance of a subset of such techniques by applying them to open source datasets, before putting them to the test on the use cases of BPCE, a major French institution in the retail banking sector. As such, we demonstrate the potential of knowledge distillation to improve the performance of these models without altering their form and simplicity.