HP-Edit: A Human-Preference Post-Training Framework for Image Editing
This work addresses the lack of scalable human-preference datasets and RLHF frameworks for diffusion-based image editing, enabling better alignment with human preferences across diverse editing tasks.
HP-Edit introduces a post-training framework for aligning image editing models with human preferences, leveraging a small amount of human-preference data and a pretrained VLM to create an automatic evaluator (HP-Scorer) and a scalable preference dataset (RealPref-50K). The approach significantly improves models like Qwen-Image-Edit-2509, aligning outputs more closely with human preference.
Common image editing tasks typically adopt powerful generative diffusion models as the leading paradigm for real-world content editing. Meanwhile, although reinforcement learning (RL) methods such as Diffusion-DPO and Flow-GRPO have further improved generation quality, efficiently applying Reinforcement Learning from Human Feedback (RLHF) to diffusion-based editing remains largely unexplored, due to a lack of scalable human-preference datasets and frameworks tailored to diverse editing needs. To fill this gap, we propose HP-Edit, a post-training framework for Human Preference-aligned Editing, and introduce RealPref-50K, a real-world dataset across eight common tasks and balancing common object editing. Specifically, HP-Edit leverages a small amount of human-preference scoring data and a pretrained visual large language model (VLM) to develop HP-Scorer--an automatic, human preference-aligned evaluator. We then use HP-Scorer both to efficiently build a scalable preference dataset and to serve as the reward function for post-training the editing model. We also introduce RealPref-Bench, a benchmark for evaluating real-world editing performance. Extensive experiments demonstrate that our approach significantly enhances models such as Qwen-Image-Edit-2509, aligning their outputs more closely with human preference.