Evaluating a Generative Adversarial Framework for Information Retrieval
This work addresses shortcomings in a popular GAN-based IR method, offering incremental improvements for researchers in information retrieval.
The paper critically analyzes the IRGAN framework for information retrieval, identifying issues with its policy gradients and generator that harm performance, and proposes two new models that outperform IRGAN on two out of three tasks.
Recent advances in Generative Adversarial Networks (GANs) have resulted in its widespread applications to multiple domains. A recent model, IRGAN, applies this framework to Information Retrieval (IR) and has gained significant attention over the last few years. In this focused work, we critically analyze multiple components of IRGAN, while providing experimental and theoretical evidence of some of its shortcomings. Specifically, we identify issues with the constant baseline term in the policy gradients optimization and show that the generator harms IRGAN's performance. Motivated by our findings, we propose two models influenced by self-contrastive estimation and co-training which outperform IRGAN on two out of the three tasks considered.