An Evaluation Study of Generative Adversarial Networks for Collaborative Filtering
This is an incremental study that critically examines the reproducibility and practical effectiveness of existing GAN-based recommendation methods for researchers and practitioners.
This work evaluated the reproducibility of CFGAN and related models for collaborative filtering recommendation systems, finding that CFGAN could degenerate into a simple autoencoder and was not consistently competitive against properly optimized baselines despite high computational cost.
This work explores the reproducibility of CFGAN. CFGAN and its family of models (TagRec, MTPR, and CRGAN) learn to generate personalized and fake-but-realistic rankings of preferences for top-N recommendations by using previous interactions. This work successfully replicates the results published in the original paper and discusses the impact of certain differences between the CFGAN framework and the model used in the original evaluation. The absence of random noise and the use of real user profiles as condition vectors leaves the generator prone to learn a degenerate solution in which the output vector is identical to the input vector, therefore, behaving essentially as a simple autoencoder. The work further expands the experimental analysis comparing CFGAN against a selection of simple and well-known properly optimized baselines, observing that CFGAN is not consistently competitive against them despite its high computational cost. To ensure the reproducibility of these analyses, this work describes the experimental methodology and publishes all datasets and source code.