Serial Speakers: a Dataset of TV Series
This dataset addresses a gap for researchers in multimedia and speech processing by enabling realistic investigations, though it is incremental as it builds on existing annotation methods for new content.
The authors tackled the lack of annotated datasets for continuous TV series by creating Serial Speakers, a dataset of 161 episodes from three popular American TV serials, providing annotations for speech turns, scene boundaries, and other multimedia features.
For over a decade, TV series have been drawing increasing interest, both from the audience and from various academic fields. But while most viewers are hooked on the continuous plots of TV serials, the few annotated datasets available to researchers focus on standalone episodes of classical TV series. We aim at filling this gap by providing the multimedia/speech processing communities with Serial Speakers, an annotated dataset of 161 episodes from three popular American TV serials: Breaking Bad, Game of Thrones and House of Cards. Serial Speakers is suitable both for investigating multimedia retrieval in realistic use case scenarios, and for addressing lower level speech related tasks in especially challenging conditions. We publicly release annotations for every speech turn (boundaries, speaker) and scene boundary, along with annotations for shot boundaries, recurring shots, and interacting speakers in a subset of episodes. Because of copyright restrictions, the textual content of the speech turns is encrypted in the public version of the dataset, but we provide the users with a simple online tool to recover the plain text from their own subtitle files.