Presenters

Ir. Ilse Ravyse - ETRO-VUB

Abstract

Communication of the face motion is an important aspect of the human interaction. Its automatic extraction from an image sequence is a challenging task due to the complexity of the face gestures. It requires the measurement and modeling of all spatial-temporal semantical properties of the face in the scene. In this dissertation, we propose the estimation of the face motion by the aid of a virtual face model that contains the related a-priori face knowledge. The sublime combination of computer vision, mechanical laws and computer graphics allows not only to analyze the rigid and nonrigid face motion, but at the same time it is capable to re-create the expressions on the 3 dimensional face model. Furthermore, the next generation video-compression standard MPEG4 intends to represent a face movie via a similar parameterized face model and foresees an efficient storage and transmission for it.

In this context, we have developed a Facial Analysis and Synthesis Scheme that accomplishes the extraction of the parameters of the 3-dimensional face motion from a 2- dimensional recording of an actor’s performance. The natural modelling of facial color, shape and apparent motion permits the retrieval of the valuable face gesture information lost in the 3- to 2-dimensional conversion inherent to the imaging process. This inverse problem has been solved at different levels:

• Face extraction and head tracking exploit the facial color and spatial properties. In the first image frame of a sequence, an ellipsoid head region is found by a personalized skin color detection and a shape-based face verification. Then, to follow this region in time, a tracking algorithm has been developed that models the joint probability of the location and the color of the detected face and that can deal with scale changes. This new method achieves a better robustness than the color-histogram trackers.


• Automatic gesture analysis is provided by the image filtering of a mathematical morphology scale-space. The information about the eye or mouth opening, contained in the local scale images, is assembled in shape control parameter. It is a fast technique of that can observe the evolution of the facial features.


• Scene calibration prepares the face model geometry to supports the motion. Using the semantic correspondences between the face image and the model, a camera calibration positions the model in the 3-dimensional face scene, and a structure calibration makes the model adaptations to person’s looks. The selection of the face morphing algorithm which does not distort the face model has been based on a new sensitivity study of the radial basis function interpolation.


• The estimation of the 3-dimensional facial expressions is an ill-posed problem due to the high degree of deformation of the face surface. It cannot be solved without imposing constraints on the face motion. In this thesis, a new estimation approach integrates a natural motion model using the muscle forces caused by displacements (implemented using a finite element model). We took into account the motion projection by linking the optical flow measurement with the modelled scene flow in a nonlinear least-squares formulation, and by giving a physical motivation to the regularization. Smooth deformations of the face model have been obtained.


• Graphically represented face animations benefit from control rules which can be easily applied to a face model. We have realized this by the dynamic application of the MPEG4 face animation parameters and their calculation from the extracted displacements.

Animating the face model with the extracted parameters, results in a faithful reproduction of the gestures. Applications in the emerging media which convey this synthetic content via a virtual human are numerous, amongst them are the interactive television, games, telepresence, news-reading on internet and intuitive user-interfaces.