Rethinking Search: Making Domain Experts out of Dilettantes
This addresses the need for scalable, reliable information retrieval for users seeking expert advice, though it appears incremental by combining existing approaches.
The paper tackles the problem of search engines and language models failing to provide expert-level, justified answers to user queries, proposing a synthesis of classical information retrieval and pre-trained language models to create systems that act as domain experts.
When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than domain experts -- they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of domain expert advice.