PAFT: Preservation Aware Fine-Tuning for Minimal-Edit Program Repair
This addresses the issue of increased review and maintenance costs for developers by producing smaller, more localized patches, though it is incremental as it builds on existing fine-tuning methods.
The paper tackles the problem of over-editing in automated program repair with large language models, where patches rewrite more code than necessary, and proposes PAFT, a preservation-aware fine-tuning method that improves pass@1 by up to 65.6% and reduces average edit distance by up to 32.6% compared to standard fine-tuning.
Large language models (LLMs) are effective for automated program repair, but plausible patches that pass the full test suite often rewrite more code than necessary, increasing review and maintenance costs. This over-editing is common because most bugs are localized, while standard supervised fine-tuning provides no explicit signal about which tokens should be preserved and which should be changed. We propose PAFT, a preservation-aware fine-tuning method for minimal-edit program repair. PAFT derives token-level preservation signals by aligning buggy and fixed code, combines them with full-sequence masking, and applies an edit-difficulty curriculum. Across Defects4J and HumanEval-Java, PAFT improves pass@1 by up to 65.6% over standard supervised fine-tuning (StdFT) while reducing average edit distance (AED) by up to 32.6%. On Defects4J with DeepSeek-Coder-6.7B, PAFT also outperforms AdaPatcher, a strong preference-based repair baseline, improving pass@1 from 5.9% to 10.1% while reducing median AED from 61.0 to 42.0. Overall, PAFT preserves stable context and concentrates edits on faulty regions, yielding smaller, more localized, plausible patches without inference-time search, reranking, or post-processing.