cTBLS: Augmenting Large Language Models with Conversational Tables
This work addresses the challenge of grounding conversational AI in structured data to reduce hallucinations, representing a strong specific gain in domain-specific applications.
The paper tackled the problem of hallucinations and accuracy in open-domain conversational LLMs by introducing cTBLS, a three-step architecture that augments LLMs with tabular information, achieving up to 125% relative improvement in retrieval and 2x improvement in ROUGE scores over previous state-of-the-art.
Optimizing accuracy and performance while eliminating hallucinations of open-domain conversational large language models (LLMs) is an open research challenge. A particularly promising direction is to augment and ground LLMs with information from structured sources. This paper introduces Conversational Tables (cTBLS), a three-step architecture to retrieve and generate dialogue responses grounded on retrieved tabular information. cTBLS uses Transformer encoder embeddings for Dense Table Retrieval and obtains up to 125% relative improvement over the retriever in the previous state-of-the-art system on the HyrbiDialogue dataset. cTBLS then uses a shared process between encoder and decoder models to perform a coarse+fine tabular knowledge (e.g., cell) ranking combined with a GPT-3.5 LLM response generator to yield a 2x relative improvement in ROUGE scores. Finally, human evaluators prefer cTBLs +80% of the time (coherency, fluency) and judge informativeness to be 4x better than the previous state-of-the-art.