SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketches
This addresses the tedious, time-consuming aspects of 2D animation production for game developers and animators, though it appears incremental relative to broader video synthesis.
The paper tackles the problem of automating repetitive 2D sprite animation by proposing a workflow-aligned approach that maps keyframes and sketches to rendered animations, demonstrating that their model outperforms an existing method.
2D animation is a common factor in game development, used for characters, effects and background art. It involves work that takes both skill and time, but parts of which are repetitive and tedious. Automated animation approaches exist, but are designed without animators in mind. The focus is heavily on real-life video, which follows strict laws of how objects move, and does not account for the stylistic movement often present in 2D animation. We propose a problem formulation that more closely adheres to the standard workflow of animation. We also demonstrate a model, SketchBetween, which learns to map between keyframes and sketched in-betweens to rendered sprite animations. We demonstrate that our problem formulation provides the required information for the task and that our model outperforms an existing method.