R. Raljundi, P. Chakravarty (equal contribution) and T. Tuytelaars. Who’s that Actor? Automatic Labelling of Actors in TV series starting from IMDB Images. Asian Conference on Computer Vision (ACCV), Taiwan, November 2016.
In this work, we aim at automatically labeling actors in a TV series.
Rather than relying on transcripts and subtitles, as has been demonstrated in the past, we show how to achieve this goal starting from a set of example images of each of the main actors involved, collected from the Internet Movie Database (IMDB).
The problem then becomes one of domain adaptation:
actors’ IMDB photos are typically taken at awards ceremonies and are quite different from their appearances in TV series. In each series as well, there is considerable change in actor appearance due to makeup, lighting, ageing, etc.
To bridge this gap, we propose a graph-matching based self-labelling algorithm, which we coin HSL (Hungarian Self Labelling). Further, we propose a new metric to be used in this context, as well as an extension that is more robust to outliers, where prototypical faces for each of the actors are selected based on a hierarchical clustering procedure. We conduct experiments with 15 episodes from 3 different TV series and demonstrate automatic annotation with an accuracy of 90% and up.
The Dataset used for this paper is available here.
The dataset consists of face track information and actor label annotations for 6 episodes of Breaking Bad, 3 episodes of Mad Men and 6 episodes of Big Bang Theory. We encourage everyone to buy the relevant DVDs and rip the annotated episodes to obtain the video material. Our annotations can then be used with your video files. If this is not possible, please print, fill out and sign this form (in which you promise to use the video data only for research purposes). Send us an e-mail with a scan of the completed form in attachment.
Poster (click to enlarge):