A Diversity-Enhanced Knowledge Distillation Model for Practical Math Word Problem Solving
This work addresses a generalization issue in math word problem solving for natural language processing applications, but it is incremental as it builds on existing knowledge distillation and variational auto-encoder methods.
The paper tackled the problem of limited diversity in solution equations for math word problem solving, introducing a Diversity-enhanced Knowledge Distillation model that achieved higher answer accuracy on four benchmark datasets.
Math Word Problem (MWP) solving is a critical task in natural language processing, has garnered significant research interest in recent years. Various recent studies heavily rely on Seq2Seq models and their extensions (e.g., Seq2Tree and Graph2Tree) to generate mathematical equations. While effective, these models struggle to generate diverse but counterpart solution equations, limiting their generalization across various math problem scenarios. In this paper, we introduce a novel Diversity-enhanced Knowledge Distillation (DivKD) model for practical MWP solving. Our approach proposes an adaptive diversity distillation method, in which a student model learns diverse equations by selectively transferring high-quality knowledge from a teacher model. Additionally, we design a diversity prior-enhanced student model to better capture the diversity distribution of equations by incorporating a conditional variational auto-encoder. Extensive experiments on {four} MWP benchmark datasets demonstrate that our approach achieves higher answer accuracy than strong baselines while maintaining high efficiency for practical applications.