Dynamic Character Graph via Online Face Clustering for Movie Analysis
This work addresses movie analysis for researchers and industry by providing a dynamic graph method, though it is incremental as it builds on existing static graph approaches.
The paper tackles the problem of automated movie content analysis by proposing an unsupervised approach to build a dynamic character graph that captures temporal evolution of character interactions, achieving superior performance in narrative structure segmentation and major character retrieval on movies with over 5000 face tracks.
An effective approach to automated movie content analysis involves building a network (graph) of its characters. Existing work usually builds a static character graph to summarize the content using metadata, scripts or manual annotations. We propose an unsupervised approach to building a dynamic character graph that captures the temporal evolution of character interaction. We refer to this as the character interaction graph(CIG). Our approach has two components:(i) an online face clustering algorithm that discovers the characters in the video stream as they appear, and (ii) simultaneous creation of a CIG using the temporal dynamics of the resulting clusters. We demonstrate the usefulness of the CIG for two movie analysis tasks: narrative structure (acts) segmentation, and major character retrieval. Our evaluation on full-length movies containing more than 5000 face tracks shows that the proposed approach achieves superior performance for both the tasks.