HyMoERec: Hybrid Mixture-of-Experts for Sequential Recommendation
This work addresses the issue of heterogeneity in user behavior and item diversity for sequential recommendation systems, representing an incremental improvement over existing methods.
The paper tackles the problem of uniform treatment of user interactions and items in sequential recommendation by proposing HyMoERec, a hybrid mixture-of-experts framework that captures diverse user behavior and item complexity, resulting in consistent outperformance of state-of-the-art baselines on MovieLens-1M and Beauty datasets.
We propose HyMoERec, a novel sequential recommendation framework that addresses the limitations of uniform Position-wise Feed-Forward Networks in existing models. Current approaches treat all user interactions and items equally, overlooking the heterogeneity in user behavior patterns and diversity in item complexity. HyMoERec initially introduces a hybrid mixture-of-experts architecture that combines shared and specialized expert branches with an adaptive expert fusion mechanism for the sequential recommendation task. This design captures diverse reasoning for varied users and items while ensuring stable training. Experiments on MovieLens-1M and Beauty datasets demonstrate that HyMoERec consistently outperforms state-of-the-art baselines.