JSON Whisperer: Efficient JSON Editing with LLMs
This addresses computational inefficiency for developers and users working with JSON data through LLMs, representing an incremental improvement over existing methods.
The paper tackled the problem of inefficient JSON editing by LLMs, which regenerate entire documents for each edit, by introducing JSON Whisperer, a framework that generates diff patches to reduce token usage by 31% while maintaining edit quality within 5% of full regeneration.
Large language models (LLMs) can modify JSON documents through natural language commands, but current approaches regenerate entire structures for each edit, resulting in computational inefficiency. We present JSON Whisperer, a framework that enables LLMs to generate RFC 6902 diff patches-expressing only the necessary modifications-rather than complete documents. We identify two key challenges in patch-based editing: (1) LLMs often miss related updates when generating isolated patches, and (2) array manipulations require tracking index shifts across operations, which LLMs handle poorly. To address these issues, we introduce EASE (Explicitly Addressed Sequence Encoding), which transforms arrays into dictionaries with stable keys, eliminating index arithmetic complexities. Our evaluation shows that patch generation with EASE reduces token usage by 31% while maintaining edit quality within 5% of full regeneration with particular gains for complex instructions and list manipulations. The dataset is available at: https://github.com/emnlp2025/JSON-Whisperer/