Revisiting Bi-Encoder Neural Search: An Encoding--Searching Separation Perspective
This provides a conceptual framework for improving neural search systems, though it appears incremental as it builds on existing bi-encoder architectures.
The paper tackles performance issues in bi-encoder neural search by analyzing the encoding information bottleneck and embedding search limitations, proposing an encoding-searching separation perspective that explains root causes and suggests strategies to potentially lower training costs and improve retrieval.
This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues such as low performance on seen datasets and weak zero-shot performance on new datasets. In this paper, we analyze these issues and summarize two main critiques: the encoding information bottleneck problem and limitations of the basic assumption of embedding search. We then construct a thought experiment to logically analyze the encoding and searching operations and challenge the basic assumptions of embedding search. Building on these observations, we propose a new perspective on the bi-encoder architecture called the \textit{encoding--searching separation} perspective, which conceptually and practically separates the encoding and searching operations. This framework is applied to explain the root cause of existing issues and suggest mitigation strategies, potentially lowering training costs and improving retrieval performance. Finally, we discuss the broader implications of the ideas underlying this perspective, the new design surface it exposes, and potential research directions arising from it.