Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning
This addresses the challenge of generating aesthetically pleasing cinemagraphs for artists and content creators, though it is incremental as it builds on existing techniques.
The paper tackles the problem of automatically creating high-quality cinemagraphs from videos by using object recognition and semantic segmentation in an optimization method, resulting in visually appealing outputs as validated by a user study.
Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.