Presenter

Ms. Isabel Gonzalez [Email]

Abstract

Facial expressions play an important role in nonverbal communication. The face can show fleeting emotions or more enduring moods, even stable personality characteristics and traits. The face can also reveal a person's age, state of health, and their gender.
Facial expression recognition deals with the classification of facial motion and facial feature deformation into abstract classes that are purely based on visual information. As such, one has an objective measure of facial appearance changes that can be used for analysis of the face and/or recognition/interpretation of one’s feeling. To measure facial expressions, in this dissertation, we use the facial action coding system (FACS). FACS is a method to measure facial expressions by identifying the muscular activity underlying transient changes in facial appearances, and describes the expressions in terms of action units (AU). Manually labeling a video sequence with AUs is extremely time consuming and requires deep knowledge about face anatomy and FACS coding. The last decade there have been many efforts to automatise this process, but current available automatic systems still lack the ability to recognise well the AUs in real world settings.
State of the art feature representation in AU recognition are hand-crafted geometric and/or appearance-based. In this work, we started with the analysis of hand-crafted features to find an optimal set both for geometric- and appearance-based features, as well as their dynamics. Although, we achieve the best state of the art performance on typical databases like CK-DFAT, this approach has several disadvantages. First, it does not generalise well to new data. Secondly, the proposed features rely heavily on accurate tracked feature points, as well as the presence of a neutral face. To overcome the above mentioned limitations, we propose using unsupervised feature learning through a convolutional deep belief network, and the use of spatial pyramid matching, to construct visual AU dictionary.
With respect to classification models, apart from the classical support vector machines (SVM), in this work, we investigate the usage of extreme learning machines (ELM) as well as efficient duration modeling of the temporal behavior of AUs via the hidden semi-Markov model (HSMM). Inference and learning are efficiently addressed by providing a graphical representation for the model also in terms of a dynamic Bayesian network (DBN).
Our results suggest that, although it is not always necessary to recognise the temporal phases of an AU, our approach offers a great insight in the need of static features to detect neutral and apex phases, and the need of dynamic features to recognise the onset and offset phases of an AU. Moreover, the proposed approach is beyond state-of-art as it generalise better to new unseen, and spontaneous data, as well in the aspect of representing facial features and modeling AUs.