Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study
This work addresses retrieval efficiency for Arabic text, but it is incremental as it compares existing methods without introducing new techniques.
The study investigated the impact of three stop word lists on Arabic information retrieval, finding that a general stoplist combined with the BM25 weighting scheme yielded the best performance.
The effectiveness of three stop words lists for Arabic Information Retrieval---General Stoplist, Corpus-Based Stoplist, Combined Stoplist ---were investigated in this study. Three popular weighting schemes were examined: the inverse document frequency weight, probabilistic weighting, and statistical language modelling. The Idea is to combine the statistical approaches with linguistic approaches to reach an optimal performance, and compare their effect on retrieval. The LDC (Linguistic Data Consortium) Arabic Newswire data set was used with the Lemur Toolkit. The Best Match weighting scheme used in the Okapi retrieval system had the best overall performance of the three weighting algorithms used in the study, stoplists improved retrieval effectiveness especially when used with the BM25 weight. The overall performance of a general stoplist was better than the other two lists.