MoE-MLoRA for Multi-Domain CTR Prediction: Efficient Adaptation with Expert Specialization
This addresses multi-domain click-through rate prediction for personalized recommendation systems, presenting an incremental improvement over existing adaptation methods.
The paper tackles the problem of adapting recommendation systems to user interactions across different domains by proposing MoE-MLoRA, a mixture-of-experts framework that improves performance in large-scale datasets (e.g., +1.45 Weighed-AUC in Taobao-20) but shows limited benefits in structured datasets with low diversity.
Personalized recommendation systems must adapt to user interactions across different domains. Traditional approaches like MLoRA apply a single adaptation per domain but lack flexibility in handling diverse user behaviors. To address this, we propose MoE-MLoRA, a mixture-of-experts framework where each expert is first trained independently to specialize in its domain before a gating network is trained to weight their contributions dynamically. We evaluate MoE-MLoRA across eight CTR models on Movielens and Taobao, showing that it improves performance in large-scale, dynamic datasets (+1.45 Weighed-AUC in Taobao-20) but offers limited benefits in structured datasets with low domain diversity and sparsity. Further analysis of the number of experts per domain reveals that larger ensembles do not always improve performance, indicating the need for model-aware tuning. Our findings highlight the potential of expert-based architectures for multi-domain recommendation systems, demonstrating that task-aware specialization and adaptive gating can enhance predictive accuracy in complex environments. The implementation and code are available in our GitHub repository.