Towards Automated Movie Trailer Generation
This addresses the time-consuming and expensive process of manual trailer creation for filmmakers and studios, representing a novel application rather than an incremental improvement in core ML.
The authors tackled the problem of automating movie trailer creation by proposing Trailer Generation Transformer (TGT), a sequence-to-sequence framework that generates trailers from full movies, which significantly outperformed previous methods on comprehensive metrics.
Movie trailers are an essential tool for promoting films and attracting audiences. However, the process of creating trailers can be time-consuming and expensive. To streamline this process, we propose an automatic trailer generation framework that generates plausible trailers from a full movie by automating shot selection and composition. Our approach draws inspiration from machine translation techniques and models the movies and trailers as sequences of shots, thus formulating the trailer generation problem as a sequence-to-sequence task. We introduce Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture. TGT movie encoder is tasked with contextualizing each movie shot representation via self-attention, while the autoregressive trailer decoder predicts the feature representation of the next trailer shot, accounting for the relevance of shots' temporal order in trailers. Our TGT significantly outperforms previous methods on a comprehensive suite of metrics.