Collaborative Editable Model
This addresses the problem of rapid development and iteration for domain-specific LLMs in fields like finance, healthcare, and law, though it is incremental as it builds on existing prompt-based adaptation methods.
The paper tackles the challenge of training vertical-domain large language models (LLMs) that require large annotated data and computational resources by introducing the Collaborative Editable Model (CoEM), which uses user-contributed knowledge and interactive dialogues to inject high-value fragments via prompts, resulting in significant improvements in domain-specific generation in a financial scenario with 15k user feedback.
Vertical-domain large language models (LLMs) play a crucial role in specialized scenarios such as finance, healthcare, and law; however, their training often relies on large-scale annotated data and substantial computational resources, impeding rapid development and continuous iteration. To address these challenges, we introduce the Collaborative Editable Model (CoEM), which constructs a candidate knowledge pool from user-contributed domain snippets, leverages interactive user-model dialogues combined with user ratings and attribution analysis to pinpoint high-value knowledge fragments, and injects these fragments via in-context prompts for lightweight domain adaptation. With high-value knowledge, the LLM can generate more accurate and domain-specific content. In a financial information scenario, we collect 15k feedback from about 120 users and validate CoEM with user ratings to assess the quality of generated insights, demonstrating significant improvements in domain-specific generation while avoiding the time and compute overhead of traditional fine-tuning workflows.