The Team

Face Tracking and Recognition

Face recognition continues to be one of the important problems in computer vision. Our goal is to develop video-based face tracking and recognition algorithms that are robust to changes of pose and lighting.

Face Recognition in Art Images


Portraits are subject to several complexities such as aesthetic sensibilities of the artist or social standing of the sitter. Moreover, the number of samples available to model these effects is often limited. For robust automated face recognition, it also becomes important to model the characteristics of the artist. From a set of portraiture where the identities of subjects is known, we derive appropriate features that are based on domain knowledge of artistic renderings and learn statistical models for the distributions of the match and non-match scores, which we refer to as the portrait feature space (PFS). The features considered include well-known facial recognition attributes like local features and anthropometric distances. Thereafter, we learn which of the chosen features were emphasized in various works involving (a) same artist depicting same sitter, (b) same sitter but by different artists, (c) same artist but depicting different sitters, and we show that the knowledge of these specific choices can provide valuable information regarding the sitter and/or artist. Further, we use the learned PFS on a number of cases that have been open questions to art historians. They are usually in the form of validating two portraits as belonging to the same person. Using statistical hypothesis tests on the PFS, we provide quantitative measures of similarity for each of these questions. It is, to the best of our knowledge, the first study that applies automated face recognition technologies to the analysis of portraits of multiple subjects in various forms - paintings, death masks, sculptures.

Sample Publications

Face Tracking and Recognition in Video

In this project, we have studied the problem of robust face tracking and recognition in the presence of large pose and illumination changes. First, we developed a theoretical model for understanding the effects of lighting, motion and shape in the process of image formation, bulding upon multilinear tensor algebra. This has led to the development of robust tracking of facial features and an analysis-by-synthesis framework for face recognition from video sequences.

Sample Publications