Post-training for Efficient Communication via Convention Formation
This work addresses the inefficiency in LLM communication for applications requiring adaptive human-like interactions, though it is incremental as it builds on existing fine-tuning methods.
The paper tackled the problem that LLMs lack the ability to form communication conventions efficiently in multi-turn interactions, and through a post-training process, it achieved significantly improved convention formation abilities in LLMs across two new benchmarks.
Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions. In contrast, prior work shows that LLMs do not naturally show this behavior. We develop a post-training process to develop this ability through targeted fine-tuning on heuristically identified demonstrations of convention formation. We evaluate with two new benchmarks focused on this capability. First, we design a focused, cognitively-motivated interaction benchmark that consistently elicits strong convention formation trends in humans. Second, we create a new document-grounded reference completion task that reflects in-the-wild convention formation behavior. Our studies show significantly improved convention formation abilities in post-trained LLMs across the two evaluation methods.