LLM-Friendly Knowledge Representation for Customer Support
This work addresses the challenge of making LLMs more effective for customer support operations, though it appears incremental as it builds on existing LLM fine-tuning methods with domain-specific adaptations.
The paper tackles the problem of applying Large Language Models (LLMs) to Airbnb customer support by introducing the Intent, Context, and Action (ICA) format for knowledge representation and a synthetic data generation strategy, resulting in improved performance with enhanced accuracy and reduced manual processing time.
We propose a practical approach by integrating Large Language Models (LLMs) with a framework designed to navigate the complexities of Airbnb customer support operations. In this paper, our methodology employs a novel reformatting technique, the Intent, Context, and Action (ICA) format, which transforms policies and workflows into a structure more comprehensible to LLMs. Additionally, we develop a synthetic data generation strategy to create training data with minimal human intervention, enabling cost-effective fine-tuning of our model. Our internal experiments (not applied to Airbnb products) demonstrate that our approach of restructuring workflows and fine-tuning LLMs with synthetic data significantly enhances their performance, setting a new benchmark for their application in customer support. Our solution is not only cost-effective but also improves customer support, as evidenced by both accuracy and manual processing time evaluation metrics.