Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering
This work addresses the problem of improving accuracy and reliability in conversational AI for users, though it is incremental as it builds on existing methods by adding graph embeddings and memory modules.
The paper tackles conversational question answering by integrating large language models with graph-based reasoning to handle questions in context and evidence from heterogeneous sources, resulting in enhanced reasoning and robustness against noise on the ConvMix benchmark.
We focus on a conversational question answering task which combines the challenges of understanding questions in context and reasoning over evidence gathered from heterogeneous sources like text, knowledge graphs, tables, and infoboxes. Our method utilizes a graph structured representation to aggregate information about a question and its context (i.e., the conversation so far and evidence retrieved to find an answer), while also harnessing the reasoning and text generation capabilities of large language models (LLMs). Graph embeddings are directly injected into the LLM, bypassing the token embedding layers, and learned end-to-end by minimizing cross-entropy. Our model maintains a memory module to track and update past evidence, thus influencing the graph's structure, as the conversation evolves. Experimental results on the ConvMix benchmark(Christmann et al., 2022a) show that graph embeddings enhance the LLM's ability to reason, while the memory module provides robustness against noise and retrieval errors.