MatLM: a Matrix Formulation for Probabilistic Language Models
This work addresses implementation difficulties for researchers and practitioners in Information Retrieval using incremental improvements to existing models.
The authors tackled the challenge of implementing probabilistic language models in matrix-friendly programming languages by reformulating them into a matrix representation, resulting in the release of a Java software package called MatLM that implements models like Dirichlet smoothing and LDA-based variants.
Probabilistic language models are widely used in Information Retrieval (IR) to rank documents by the probability that they generate the query. However, the implementation of the probabilistic representations with programming languages that favor matrix calculations is challenging. In this paper, we utilize matrix representations to reformulate the probabilistic language models. The matrix representation is a superstructure for the probabilistic language models to organize the calculated probabilities and a potential formalism for standardization of language models and for further mathematical analysis. It facilitates implementations by matrix friendly programming languages. In this paper, we consider the matrix formulation of conventional language model with Dirichlet smoothing, and two language models based on Latent Dirichlet Allocation (LDA), i.e., LBDM and LDI. We release a Java software package--MatLM--implementing the proposed models. Code is available at: https://github.com/yanshanwang/JGibbLDA-v.1.0-MatLM.