RemEdit: Efficient Diffusion Editing with Riemannian Geometry
This addresses the problem of efficient and high-quality image editing for users of generative AI, representing a strong specific gain rather than a foundational advancement.
The paper tackles the trade-off between semantic fidelity and inference speed in controllable image generation by introducing RemEdit, a diffusion-based framework that uses Riemannian geometry for smooth edits and attention pruning for acceleration, achieving state-of-the-art editing performance with real-time speed under 50% pruning.
Controllable image generation is fundamental to the success of modern generative AI, yet it faces a critical trade-off between semantic fidelity and inference speed. The RemEdit diffusion-based framework addresses this trade-off with two synergistic innovations. First, for editing fidelity, we navigate the latent space as a Riemannian manifold. A mamba-based module efficiently learns the manifold's structure, enabling direct and accurate geodesic path computation for smooth semantic edits. This control is further refined by a dual-SLERP blending technique and a goal-aware prompt enrichment pass from a Vision-Language Model. Second, for additional acceleration, we introduce a novel task-specific attention pruning mechanism. A lightweight pruning head learns to retain tokens essential to the edit, enabling effective optimization without the semantic degradation common in content-agnostic approaches. RemEdit surpasses prior state-of-the-art editing frameworks while maintaining real-time performance under 50% pruning. Consequently, RemEdit establishes a new benchmark for practical and powerful image editing. Source code: https://www.github.com/eashanadhikarla/RemEdit.