Face Analysis and Emotion Recognition

MMER demonstration video.

What's in a face?

In human-human interaction the visual communication channel, i.e. the body pose, gestures and especially the face, contributes immensly to the efficiency and quality of the communication. The exchange of pure verbal content regularly leads to mis-interpretations and mis-understandings. Thus, for technical applications which want to analyse and improve human-human or human-computer communication, the analysis of the faces is mandatory. A human face itself provides various information about the person: The gender, age, ethnic origin, ametropia wearing glasses), beards, identity, plus the emotion. By using the term "emotion" we address general mental impulses such as discrete states (anger, joy, fear, sadness, etc.) and continuous scales (activity, interest level, positiveness, etc.).

FEASy - Facial Expression Analysis System

The MMER-Team of the FEASy - the Facial Expression Analysis System. It is designed to fully automatically extract all relevant information from human faces. Besides the aim of best possible accuracy, FEASy is developed with focus on real-time applications by utilizing graphics processing power and high-performance programming methods.

The Research Challenge

The task of face analysis with whatever scope is characterized by highly complicated conditions. In the first place, the appearance of faces varies between different persons in their outer shape, the relative spatial position of facial features (eyes, brows, nose, mouth, chin), the shape of facial features, and the skin color. Due to the high complexity and variance of face recognition and interpretaion, the evolution even dedicated an own area of the human visual brain cortex to this task (The Journal of Neuroscience, 1997, Wikipedia). Therefore, human face analysis constitutes one of the greatest challenges in the modern computer vision area.

FEASy utilizes the basic idea of the Active Appearance Model approach: A statistical model describes the variations in shape and texture of human faces derived from a careful selection of photographs showing different persons with different facial expression and in various lighting conditions. During the analysis of a human face within a video or a picture, the Appearance Model is used to re-synthesize this face as optimal as possible. Finally, this technology provides an extremely compact parametrization of the analyzed face by setting between 30 and 60 numeric parameters.

FEASy is a computationally optimized system consisting of a processing chain with modules for Head Localization, Eye Localization, AAM-Face Parameterization, and Classification. All these instances are built and run in MMER_Lab a generic multi-threading framework, which provides comprehensive features at optimal performance, modularity, and flexibility allowing for efficient multi-stream and asynchronous signal processing.

Applications

Human-Computer Interfaces

Our shape and texture based analysis of objects provides a most comprehensive and precise parametrization for multiple purposes. Recognition results on human faces, such as gender, age, ethnic origin, person identity, and emotion are vitally important as soon as it comes to Human-Machine Interfaces which strive to adapt to their users. This addresses the whole range from modern car-driver assistants, market research interviews, (remote) meeting assistants, gaming consoles, down to office computers and home multi-media devices. [more]

Shop window evaluation

As director of a shopping mall, a big store, or shop you would certainly be interested, if the display in your showcases attracts the attention of people walking by. FEASy can be utilized for an automatic and objective evaluation of your showcases.

Efficient Face Image Coding

Smart chip cards have an enormous market in the area of security for person identification. Codes, passwords, personal data and even face images have to be stored on them. However, the memory on such chip cards is very limited and images occupy the most. With our technology, it is possible to store a picture with approx. 480 bytes. [more]

Anonymity in video communication

The attractiveness of today's online chat rooms is mainly derived from the secure protection of the anonymity of the chat participants allowing to talk about most private issues if desired. However, the main drawback is the pure text based communication, which lead to an invented chat-language. It comprises numerous different Smileys and *xy* codes for the expression of the emotion, which is so essential for a good communication and fewer misunderstanding.

The protection of anonymity within natural human-human video communication is another application scenario of the FEASy technology. Since the face analysis of FEASy is based on the re-synthesis of a face, the found shapes and positions of eyes, brows, mouth, etc. can be applied to any photography of a face from, say, Brad Pitt, Julia Roberts, or anybody else. Thus, today's text-chat rooms can significantly benefit from providing video chat functionality with the entertainment factor of fitting e.g. celebrities' faces to each chat partner. [more]

Beyond faces

The methodology of FEASy proved its applicability to human faces, which can be seen as an object class with a highest variance in shape and texture. Thus, we believe it can be utilized in many different areas of visual quality check, medical images, object positioning, surveillance, etc.

Please feel free to contact us for a qualified feasibility statement for your application.

Contact