HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks
This addresses a critical issue for real-world applications like code editors, where faithful editing is essential to avoid breaking functionality, representing a strong specific gain in a domain-specific context.
The paper tackled the problem of instruction-based text editing in large language models, which often fail to align edits with user intents and over-edit unchanged content, by proposing HyperEdit with hypernetwork-based dynamic adaptation and difference-aware regularization, achieving a 9%–30% relative improvement in BLEU on modified regions over state-of-the-art baselines using only 3B parameters.
Instruction-based text editing is increasingly critical for real-world applications such as code editors (e.g., Cursor), but Large Language Models (LLMs) continue to struggle with this task. Unlike free-form generation, editing requires faithfully implementing user instructions while preserving unchanged content, as even minor unintended modifications can break functionality. Existing approaches treat editing as generic text generation, leading to two key failures: they struggle to faithfully align edits with diverse user intents, and they often over-edit unchanged regions. We propose HyperEdit to address both issues. First, we introduce hypernetwork-based dynamic adaptation that generates request-specific parameters, enabling the model to tailor its editing strategy to each instruction. Second, we develop difference-aware regularization that focuses supervision on modified spans, preventing over-editing while ensuring precise, minimal changes. HyperEdit achieves a 9%--30% relative improvement in BLEU on modified regions over state-of-the-art baselines, despite utilizing only 3B parameters.