InteractPro: A Unified Framework for Motion-Aware Image Composition
This addresses the challenge of dynamic image composition for applications in graphics and AI, offering a unified approach to overcome manual planning and static outputs, though it appears incremental by combining existing methods.
The paper tackles the problem of generating motion-aware image compositions by introducing InteractPro, a framework that uses a planner to select between simulation-based and diffusion-based methods, resulting in controllable and coherent outputs across varied scenarios.
We introduce InteractPro, a comprehensive framework for dynamic motion-aware image composition. At its core is InteractPlan, an intelligent planner that leverages a Large Vision Language Model (LVLM) for scenario analysis and object placement, determining the optimal composition strategy to achieve realistic motion effects. Based on each scenario, InteractPlan selects between our two specialized modules: InteractPhys and InteractMotion. InteractPhys employs an enhanced Material Point Method (MPM)-based simulation to produce physically faithful and controllable object-scene interactions, capturing diverse and abstract events that require true physical modeling. InteractMotion, in contrast, is a training-free method based on pretrained video diffusion. Traditional composition approaches suffer from two major limitations: requiring manual planning for object placement and generating static, motionless outputs. By unifying simulation-based and diffusion-based methods under planner guidance, InteractPro overcomes these challenges, ensuring richly motion-aware compositions. Extensive quantitative and qualitative evaluations demonstrate InteractPro's effectiveness in producing controllable, and coherent compositions across varied scenarios.