Evaluation of Information Retrieval Systems Using Structural Equation Modelling
This work addresses the need for better interpretation of evaluation results in information retrieval, offering a methodological improvement for system designers and experimenters, though it appears incremental as it applies an existing statistical technique to this domain.
The paper tackles the problem of interpreting experimental data from information retrieval system evaluations by using Structural Equation Modelling (SEM) to explain relationships between variables and latent factors affecting performance, providing designers and engineers with insights into system successes and failures.
The interpretation of the experimental data collected by testing systems across input datasets and model parameters is of strategic importance for system design and implementation. In particular, finding relationships between variables and detecting the latent variables affecting retrieval performance can provide designers, engineers and experimenters with useful if not necessary information about how a system is performing. This paper discusses the use of Structural Equation Modelling (SEM) in providing an in-depth explanation of evaluation results and an explanation of failures and successes of a system; in particular, we focus on the case of Information Retrieval.