Improving Contrastive Learning with Model Augmentation
This work addresses the challenge of enhancing contrastive learning in sequential recommendation for better prediction accuracy, though it appears incremental by shifting from data to model augmentation.
The paper tackles the problem of data sparsity and noise in sequential recommendation by proposing a new self-supervised learning paradigm that uses model augmentation instead of data augmentation to construct views for contrastive learning, resulting in improved performance as verified by experiments.
The sequential recommendation aims at predicting the next items in user behaviors, which can be solved by characterizing item relationships in sequences. Due to the data sparsity and noise issues in sequences, a new self-supervised learning (SSL) paradigm is proposed to improve the performance, which employs contrastive learning between positive and negative views of sequences. However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals. Therefore, we investigate the possibility of model augmentation to construct view pairs. We propose three levels of model augmentation methods: neuron masking, layer dropping, and encoder complementing. This work opens up a novel direction in constructing views for contrastive SSL. Experiments verify the efficacy of model augmentation for the SSL in the sequential recommendation. Code is available\footnote{\url{https://github.com/salesforce/SRMA}}.