Interpreting Neural Ranking Models using Grad-CAM
This addresses the challenge of explaining complex neural network outputs for researchers and practitioners in IR, though it is incremental as it applies an existing interpretability method to a new domain.
The paper tackles the problem of interpreting neural ranking models (NRMs) in information retrieval by adopting Grad-CAM to visualize how query-document term pairs contribute to matching scores, providing insights into relevance and enabling applications like snippet generation and user-query intent analysis.
Recently, applying deep neural networks in IR has become an important and timely topic. For instance, Neural Ranking Models(NRMs) have shown promising performance compared to the traditional ranking models. However, explaining the ranking results has become even more difficult with NRM due to the complex structure of neural networks. On the other hand, a great deal of research is under progress on Interpretable Machine Learning(IML), including Grad-CAM. Grad-CAM is an attribution method and it can visualize the input regions that contribute to the network's output. In this paper, we adopt Grad-CAM for interpreting the ranking results of NRM. By adopting Grad-CAM, we analyze how each query-document term pair contributes to the matching score for a given pair of query and document. The visualization results provide insights on why a certain document is relevant to the given query. Also, the results show that neural ranking model captures the subtle notion of relevance. Our interpretation method and visualization results can be used for snippet generation and user-query intent analysis.