On Estimating the Training Cost of Conversational Recommendation Systems
This addresses the problem of high training costs for developers and researchers in conversational AI, but it is incremental as it focuses on analyzing and mitigating an existing bottleneck.
The paper tackles the high computational training cost of state-of-the-art conversational recommendation systems by examining five representative strategies to demonstrate the issue and discussing knowledge distillation as a way to reduce online inference time.
Conversational recommendation systems have recently gain a lot of attention, as users can continuously interact with the system over multiple conversational turns. However, conversational recommendation systems are based on complex neural architectures, thus the training cost of such models is high. To shed light on the high computational training time of state-of-the art conversational models, we examine five representative strategies and demonstrate this issue. Furthermore, we discuss possible ways to cope with the high training cost following knowledge distillation strategies, where we detail the key challenges to reduce the online inference time of the high number of model parameters in conversational recommendation systems