Conversational Dueling Bandits in Generalized Linear Models
This work addresses the problem of ambiguous user feedback and linear reward assumptions in conversational recommendation systems, offering a more informative and practical approach, though it appears incremental by building on dueling bandits and generalized linear models.
The paper tackled the limitations of existing conversational recommendation systems by introducing relative feedback and non-linear reward structures, resulting in a novel algorithm called ConDuel that shows efficacy through theoretical regret bounds and empirical validations on synthetic and real-world data.
Conversational recommendation systems elicit user preferences by interacting with users to obtain their feedback on recommended commodities. Such systems utilize a multi-armed bandit framework to learn user preferences in an online manner and have received great success in recent years. However, existing conversational bandit methods have several limitations. First, they only enable users to provide explicit binary feedback on the recommended items or categories, leading to ambiguity in interpretation. In practice, users are usually faced with more than one choice. Relative feedback, known for its informativeness, has gained increasing popularity in recommendation system design. Moreover, current contextual bandit methods mainly work under linear reward assumptions, ignoring practical non-linear reward structures in generalized linear models. Therefore, in this paper, we introduce relative feedback-based conversations into conversational recommendation systems through the integration of dueling bandits in generalized linear models (GLM) and propose a novel conversational dueling bandit algorithm called ConDuel. Theoretical analyses of regret upper bounds and empirical validations on synthetic and real-world data underscore ConDuel's efficacy. We also demonstrate the potential to extend our algorithm to multinomial logit bandits with theoretical and experimental guarantees, which further proves the applicability of the proposed framework.