Face recognition has existed as a field of research for more than 30 years and has been particularly active since the early 1990s. Researchers of many different fields (from psychology, pattern recognition, neuroscience, computer graphics and computer vision) have attempted to create and understand face recognition systems. One of the most challenging tasks in automated face recognition is the matching between face images acquired in heterogeneous environments. Use-cases can cover matching of faces in unconstrained scenarios (e.g. at a distance), with long time lapse between the probe and the gallery and faces sensed in different modalities, such as thermal infrared or near infrared images (NIR) against visible spectra images (VIS). Successful solutions to heterogeneous face recognition can extend the reach of these systems to covert scenarios, such as recognition at a distance or at nighttime, or even in situations where no face even exists (forensic sketch recognition). The key difficult in matching faces from heterogeneous conditions is that images of the same subject may differ in appearance due to changes in image modality (e.g. between VIS images and NIR images, between VIS images and sketches images) introducing high intra-class variations. With these variations, a direct comparison between samples generally results in poor matching accuracy. Heterogeneous face recognition algorithms must develop facial representations invariant to these changes. In this proposal, we present three strategies to cope with these challenges. We will start by investigating manifold learning techniques. The key idea of manifold learning is to learn a joint mapping that project images, of different modalities, into a subspace where these projections can be compared directly with a simple distance function.
As a second step, we will investigate generative approaches for face recognition. Generative approaches compute the likelihood of an observation (face image) or a set of observations given the a statistical model of the subject. In this environment, cross-sensor or hetereogeneous matching can be regarded as a session variability information the model needs to cope with, compensating for them during the enrollment and probing. Finally, a combination of both techniques will be investigated. This combination can be carried out in two different ways. At first, we will investigate score level fusion between the joint map learned with manifold learning approaches and the generative approach. We will then investigate the possibility to provide face images projected with the
manifold learning techniques as input the generative face recognition systems.