Boolean-aware Attention for Dense Retrieval
This addresses a domain-specific problem for information retrieval systems that need to handle Boolean queries more effectively.
The paper tackles the problem of processing Boolean queries in dense retrieval by introducing Boolean-aware attention, which dynamically adjusts token focus based on Boolean operators. The result shows that integrating this mechanism with BERT greatly enhances the model's capability on Boolean retrieval datasets.
We present Boolean-aware attention, a novel attention mechanism that dynamically adjusts token focus based on Boolean operators (e.g., and, or, not). Our model employs specialized Boolean experts, each tailored to amplify or suppress attention for operator-specific contexts. A predefined gating mechanism activates the corresponding experts based on the detected Boolean type. Experiments on Boolean retrieval datasets demonstrate that integrating BoolAttn with BERT greatly enhances the model's capability to process Boolean queries.