ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding
This addresses the challenge of controllable video retake generation for dynamically captured videos, representing an incremental improvement over prior methods.
The paper tackles the problem of generating variable-length video retakes with camera control by introducing ReDirector, which corrects RoPE misuse and adds Rotary Camera Encoding to integrate multi-view relationships. This approach improves dynamic object localization and background preservation, achieving significant gains in camera controllability, geometric consistency, and video quality across diverse trajectories and lengths.
We present ReDirector, a novel camera-controlled video retake generation method for dynamically captured variable-length videos. In particular, we rectify a common misuse of RoPE in previous works by aligning the spatiotemporal positions of the input video and the target retake. Moreover, we introduce Rotary Camera Encoding (RoCE), a camera-conditioned RoPE phase shift that captures and integrates multi-view relationships within and across the input and target videos. By integrating camera conditions into RoPE, our method generalizes to out-of-distribution camera trajectories and video lengths, yielding improved dynamic object localization and static background preservation. Extensive experiments further demonstrate significant improvements in camera controllability, geometric consistency, and video quality across various trajectories and lengths.