Document Retrieval for Large Scale Content Analysis using Contextualized Dictionaries
This addresses a specific challenge in document retrieval for social science researchers dealing with abstract topics, but it is incremental as it builds on existing topic models and co-occurrence techniques.
The paper tackles the problem of retrieving relevant documents for content analysis in social sciences when analysts cannot specify key terms, by using manually compiled reference documents to create large queries called dictionaries. The method improves retrieval results compared to alternative key term extraction methods and ignoring co-occurrence data, as shown in evaluations.
This paper presents a procedure to retrieve subsets of relevant documents from large text collections for Content Analysis, e.g. in social sciences. Document retrieval for this purpose needs to take account of the fact that analysts often cannot describe their research objective with a small set of key terms, especially when dealing with theoretical or rather abstract research interests. Instead, it is much easier to define a set of paradigmatic documents which reflect topics of interest as well as targeted manner of speech. Thus, in contrast to classic information retrieval tasks we employ manually compiled collections of reference documents to compose large queries of several hundred key terms, called dictionaries. We extract dictionaries via Topic Models and also use co-occurrence data from reference collections. Evaluations show that the procedure improves retrieval results for this purpose compared to alternative methods of key term extraction as well as neglecting co-occurrence data.