Graph-enhanced Optimizers for Structure-aware Recommendation Embedding Evolution
This work addresses the challenge of effectively utilizing graph structural information in recommender systems, offering a novel technical route beyond explicit GNN modules, though it appears incremental as an enhancement to existing optimizers.
The paper tackles the problem of embedding evolution in recommender systems by proposing a structure-aware embedding update mechanism (SEvo) that directly injects graph structural information into embeddings with minimal computational overhead. The result is consistent performance improvements across various models and datasets when integrated with optimizers like AdamW.
Embedding plays a key role in modern recommender systems because they are virtual representations of real-world entities and the foundation for subsequent decision-making models. In this paper, we propose a novel embedding update mechanism, Structure-aware Embedding Evolution (SEvo for short), to encourage related nodes to evolve similarly at each step. Unlike GNN (Graph Neural Network) that typically serves as an intermediate module, SEvo is able to directly inject graph structural information into embedding with minimal computational overhead during training. The convergence properties of SEvo along with its potential variants are theoretically analyzed to justify the validity of the designs. Moreover, SEvo can be seamlessly integrated into existing optimizers for state-of-the-art performance. Particularly SEvo-enhanced AdamW with moment estimate correction demonstrates consistent improvements across a spectrum of models and datasets, suggesting a novel technical route to effectively utilize graph structural information beyond explicit GNN modules.