CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents
This work addresses the challenge of accessing non-English scholarly content for researchers and users, though it is incremental as it builds on existing retrieval methods with a new dataset.
The authors tackled the problem of cross-lingual information retrieval for scientific documents by creating CLIRudit, a new English-French academic retrieval dataset, and found that dense embeddings without translation perform nearly as well as systems using machine translation, with document translation being more effective than query translation.
Cross-lingual information retrieval (CLIR) helps users find documents in languages different from their queries. This is especially important in academic search, where key research is often published in non-English languages. We present CLIRudit, a novel English-French academic retrieval dataset built from Érudit, a Canadian publishing platform. Using multilingual metadata, we pair English author-written keywords as queries with non-English abstracts as target documents, a method that can be applied to other languages and repositories. We benchmark various first-stage sparse and dense retrievers, with and without machine translation. We find that dense embeddings without translation perform nearly as well as systems using machine translation, that translating documents is generally more effective than translating queries, and that sparse retrievers with document translation remain competitive while offering greater efficiency. Along with releasing the first English-French academic retrieval dataset, we provide a reproducible benchmarking method to improve access to non-English scholarly content.