MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries
This addresses a practical problem for bilingual web searchers and the IR community by providing a benchmark, though it is incremental as it builds on existing multilingual models.
The authors tackled the lack of research on mixed-language queries in web searches by introducing MiLQ, the first public benchmark for such queries, and found that multilingual IR models perform moderately on it while showing inconsistent results across query types, with intentional English mixing proving effective for searching English documents due to enhanced token matching.
Despite bilingual speakers frequently using mixed-language queries in web searches, Information Retrieval (IR) research on them remains scarce. To address this, we introduce MiLQ, Mixed-Language Query test set, the first public benchmark of mixed-language queries, qualified as realistic and relatively preferred. Experiments show that multilingual IR models perform moderately on MiLQ and inconsistently across native, English, and mixed-language queries, also suggesting code-switched training data's potential for robust IR models handling such queries. Meanwhile, intentional English mixing in queries proves an effective strategy for bilinguals searching English documents, which our analysis attributes to enhanced token matching compared to native queries.