Facial Action Coding System

In this paper, Deterministic Cellular Automata (DCA) based video shot classification and retrieval is proposed. The deterministic 2D Cellular automata model captures the human facial expressions, both spontaneous and posed. The determinism stems from the fact that the facial muscle actions are standardized by the encodings of Facial Action Coding System (FACS) and Action Units (AUs). Based on these encodings, we generate the set of evolutionary update rules of the DCA for each facial expression. We consider a Person-Independent Facial Expression Space (PIFES) to analyze the facial expressions based on Partitioned 2D-Cellular Automata which capture the dynamics of facial expressions and classify the shots based on it. Target video shot is retrieved by comparing the similar expression is obtained for the query frame's face with respect to the key faces expressions in the database video. Consecutive key face expressions in the database that are highly similar to the query frame's face, then the key faces are used to generate the set of retrieved video shots from the database. A concrete example of its application which realizes an affective interaction between the computer and the user is proposed. In the affective interaction, the computer can recognize the facial expression of any given video shot. This interaction endows the computer with certain ability to adapt to the user's feedback.

Independently, we created descriptive systems to characterize chimpanzee facial behavior, responding to a common need to have an objective, standardized coding system to ask questions about primate facial behaviors. Even with slightly different systems, we arrive at similar outcomes, with convergent conclusions about chimpanzee facial mobility. This convergence is a validation of the importance of the approach, and provides support for the future use of a facial action coding system for chimpanzees, ChimpFACS. Chimpanzees ...

A framework for generating facial expressions from emotional states in daily conversation is described. It provides a mapping between emotional states and facial expressions, where the former is represented by vectors with psychologically-defined abstract dimensions, and the latter is coded by the Facial Action Coding System. In order to obtain the mapping, parallel data with rated emotional states and facial

We present a robust method to map detected facial Action Units (AUs) to six basic emotions. Automatic AU recognition is prone to errors due to illumination, tracking failures and occlusions. Hence, traditional rule based methods to map AUs to emotions are very ...

Facial expression has been a focus of emotion research for over a hundred years (Darwin, 1872/1998). It is central to several leading theories of emotion (Ekman, 1992; Izard, 1977; Tomkins, 1962) and has been the focus of at times heated debate about issues in emotion science (Ekman, 1973, 1993; Fridlund, 1992; Russell, 1994). Facial expression figures prominently in research on almost every aspect of emotion, including psychophysiology (Levenson, Ekman, & Friesen, 1990), neural bases (Calder et al., 1996; Davidson, Ekman, ...

A psychiatrist was filmed, interviewing 59 patients who had just made a suicide attempt. Within two years, 11 of these patients made one more suicide attempt. This group was matched with a similar group of 12 patients who did not reattempt within the two years. Film samples of the facial behaviour of the two groups of patients and of the doctor were coded, using Ekman and Friesen's Facial Action Coding System (FACS), with the aim of identifying bodily signs that could be associ- ated with reattempt risk. We found a number of such signs, in both the doctor and the patients, which discriminated more than 80% (18) of the 23 interviews in function of the patient's suicide reattempt risk.

The computer vision field has advanced to the point that we are now able to begin to apply automatic facial expression recognition systems to important research questions in behavioral science. The machine perception lab at UC San Diego has developed a system based on machine learning for fully automated detection of 30 actions from the facial action coding system (FACS). The system, called Computer Expression Recognition Toolbox (CERT), operates in real-time and is robust to the video conditions in real applications. This paper describes two experiments which are the first applications of this system to analyzing spontaneous human behavior: Automated discrimination of posed from genuine expressions of pain, and automated detection of driver drowsiness. The analysis revealed information about facial behavior during these conditions that were previously unknown, including the coupling of movements. Automated classifiers were able to differentiate real from fake pain significantly better than naïve human subjects, and to detect critical drowsiness above 98% accuracy.  Issues for application of machine learning systems to facial expression analysis are discussed.

The computer vision field has advanced to the point that we are now able to begin to apply automatic facial expression recognition systems to important research questions in behavioral science. The machine perception lab at UC San Diego has developed a system based on machine learning for fully automated detection of 30 actions from the facial action coding system (FACS). The system, called Computer Expression Recognition Toolbox (CERT), operates in real-time and is robust to the video conditions in real applications. This paper describes two experiments which are the first applications of this system to analyzing spontaneous human behavior: Automated discrimination of posed from genuine expressions of pain, and automated detection of driver drowsiness. The analysis revealed information about facial behavior during these conditions that were previously unknown, including the coupling of movements. Automated classifiers were able to differentiate real from fake pain significantly better than naïve human subjects, and to detect critical drowsiness above 98% accuracy.  Issues for application of machine learning systems to facial expression analysis are discussed.

The advance of computing technology has provided the means for building intelligent vehicle systems. Drowsy driver detection system is one of the potential applications of intelligent vehicle systems. Previous approaches to drowsiness detection primarily make pre-assumptions about the relevant behavior, focusing on blink rate, eye closure, and yawning. Here we employ machine learning to datamine actual human behavior during drowsiness episodes. Automatic classifiers for 30 facial actions from the Facial Action Coding system were developed using machine learning on a separate database of spontaneous expressions. These facial actions include blinking and yawn motions, as well as a number of other facial movements. In addition, head motion was collected through automatic eye tracking and an accelerometer. These measures were passed to learning-based classifiers such as Adaboost and multinomial ridge regression. The system was able to predict sleep and crash episodes during a driving computer game with 96% accuracy within subjects and above 90% accuracy across subjects. This is the highest prediction rate reported to date for detecting real drowsiness. Moreover, the analysis revealed new information about human behavior during drowsy driving.

We describe a computer vision system for observing the" action units" of a face using video sequences as input. The visual observation (sensing) is achieved by using anoptimal estimation optical flow method coupled with a geometricand a physical (muscle) model describing the facialstructure. This modeling results in a time-varying spatialpatterning of facial shape and a parametric representationof the independent muscle action groups, responsible forthe observed facial motions. These muscle action patternsmay then be ...

Impaired facial expressions of emotions have been described as characteristic symptoms of schizophrenia. Differences regarding individual facial muscle changes associated with specific emotions in posed and evoked expressions remain unclear. This study examined static facial expressions of emotions for evidence of flattened and inappropriate affect in persons with stable schizophrenia. 12 persons with stable schizophrenia and matched healthy controls underwent a standardized procedure for posed and evoked facial expressions of five universal emotions, including happy, sad, anger, fear, and disgust expressions, at three intensity levels. Subjects completed self-ratings of their emotion experience. Certified raters coded images of facial expressions for presence of action units (AUs) according to the Facial Action Coding System. Logistic regression analyses were used to examine differences in the presence of AUs and emotion experience ratings by diagnosis, condition and intensity of e...

Impaired facial expressions of emotions have been described as characteristic symptoms of schizophrenia. Differences regarding individual facial muscle changes associated with specific emotions in posed and evoked expressions remain unclear. This study examined static facial expressions of emotions for evidence of flattened and inappropriate affect in persons with stable schizophrenia. 12 persons with stable schizophrenia and matched healthy controls underwent a standardized procedure for posed and evoked facial expressions of five universal emotions, including happy, sad, anger, fear, and disgust expressions, at three intensity levels. Subjects completed self-ratings of their emotion experience. Certified raters coded images of facial expressions for presence of action units (AUs) according to the Facial Action Coding System. Logistic regression analyses were used to examine differences in the presence of AUs and emotion experience ratings by diagnosis, condition and intensity of e...

This study examines the development of children's ability to express emotions in their human figure drawing. Sixty children of 5, 8, and 11 years were asked to draw "a man," and then a "sad", "happy," "angry" and "surprised" man. Expressivity of the drawings was assessed by means of two procedures: a limited choice and a free labelling procedure. Emotionally expressive drawings were then evaluated in terms of the number and the type of graphic cues that were used to express emotion. It was found that children are able to depict happiness and sadness at 8, anger and surprise at 11. With age, children use increasingly numerous and complex graphic cues for each emotion (i.e., facial expression, body position, and contextual cues). Graphic cues for facial expression (e.g., concave mouth, curved eyebrows, wide opened eyes) share strong similarities with specific "action units" described by Ekman and Friesen (1978) in their Facial Action Coding System. Children's ability to depict emotion in their human figure drawing is discussed in relation to perceptual, conceptual, and graphic abilities.