Executive Control of Facial Expressions of Emotion

Abstract

Automated assessment with computer software offers new, efficient opportunities for measurement of facial expressions of emotion. However, little is known about reliability and validity of these assessment tools. In Publication 1, we investigated the quality of a machine vision algorithm (FACET, iMotions, 2016) using standardized databases of dynamic facial expressions under various conditions (angle, distance, lighting, and resolution). We found high reliability in ratings concordance for facial expressions and went on examining the convergent validity of automated assessment and electromyography (EMG) by measuring reaction times (RTs) during the production of joy and anger expressions in a response priming task. Both EMG and automated assessment data showed similar performance costs in RTs when inhibiting an incorrectly prepared expression and reprogramming the correct one. These results support the use of automated assessment for evaluating experimental effects in facial expressions. In Publication 2, we combined electroencephalography (EEG) and automated facial expression assessment, for the first time. We started examining facial expressions of joy, fear, and disgust in response to different visual stimuli using a go/no-go task. Then we went on focusing on expressions of joy and disgust influenced by gaze direction (with and without eye contact) in a more natural setting with a real person as the stimulus. Analysis of RTs, errors, and an event related potential (ERP) analysis of the no-go P3 suggest that facial expressions modulated by mimicry, emotional reactions, and push factors require greater top-down control, especially for expressions of joy compared to fear and disgust. In Publication 3, we examine the demands of a social situation using a Stroop-like task. Participants took part in a simulated online dating scenario to study possible moderation effects FACIAL EXPRESSION OF EMOTION 6 of perceived attractiveness on facial expressions. RTs showed facilitation for deliberate expressions of joy and disgust with congruent pictures, but interference from incongruent expressions only affected joy. Accuracy decreased when participants expressed disgust at a smiling, attractive stimulus, without affecting RTs. ERP data revealed an early posterior negativity (EPN) for better sensory processing of joy over neutral expressions and a more negative N2 indicating conflict detection from mismatched expressions. Our findings demonstrate automatic tendencies to imitate facial expressions of joy and disgust. Across all experiments, we successfully utilized and demonstrated the benefits of an automated assessment technique for investigating the executive control of facial expressions. However, the lack of process theory of emotion and the experimental study designs, with their numerous trial repetitions, suggest a lack of construct validity, raising doubts about the involvement of emotions in the observed effects.

Keywords: Facial action coding system, Automated facial expression recognition, Reliability, Validity, Executive functions, Top-down control, Go/no-go task, Emotional expression interference, Facial mimicry