Video Face Clustering with Unknown Number of Clusters
This addresses the challenge of character identification in videos like TV series for video analysis applications, but it is incremental as it adapts existing metric learning methods to a specific setting.
The paper tackles the problem of clustering face tracks in videos without prior knowledge of the number of characters, including minor ones, by proposing Ball Cluster Learning (BCL), which carves the embedding space into equal-sized balls to estimate clusters and achieves promising results on standard datasets.
Understanding videos such as TV series and movies requires analyzing who the characters are and what they are doing. We address the challenging problem of clustering face tracks based on their identity. Different from previous work in this area, we choose to operate in a realistic and difficult setting where: (i) the number of characters is not known a priori; and (ii) face tracks belonging to minor or background characters are not discarded. To this end, we propose Ball Cluster Learning (BCL), a supervised approach to carve the embedding space into balls of equal size, one for each cluster. The learned ball radius is easily translated to a stopping criterion for iterative merging algorithms. This gives BCL the ability to estimate the number of clusters as well as their assignment, achieving promising results on commonly used datasets. We also present a thorough discussion of how existing metric learning literature can be adapted for this task.