Ohio State University researchers have taught computers how to recognize 21 different human emotions from distinct facial expressions. If you did not realize there are that many feelings in our emotional vocabulary, however, then you may have forgotten about seemingly contradictory emotions, such as “happily disgusted” or “sadly angry.”

Experts have been working on decoding the mechanisms that allow our faces to express emotion since at least the time of Aristotle and the Physiognomonics treatise. Cognitive scientists today are interested in tracing the origins of distinct facial expressions back to the genes, chemicals and neural pathways that cause our brains to experience emotion.

However, these scientists have largely restricted their investigations to six primary emotions – happy, sad, fearful, angry, surprised and disgusted. This is because the facial expressions associated with these emotions were considered to be self evident.

But according to co-author of the new study, Aleix Martinez, cognitive scientist and associate professor of electrical and computer engineering at Ohio State, the problem of applying this analytical approach to an individual subject is that it is like painting a portrait with only primary colors. “It can provide an abstracted image of the person,” Prof. Martinez claims, “but not a true-to-life one.”

montage of different facial expressionsShare on Pinterest
Volunteers were asked to provide appropriate facial expressions to verbal cues such as “you just got some great unexpected news” or “you smell a bad odor.”
Image credit: The Ohio State University

By mixing different combinations of emotions and coding their associated facial expressions, Prof. Martinez has more than tripled the “palette” of human emotions that can be understood by a computer and, therefore, examined through “rigorous scientific study.”

He refers to these combined emotions as “compound emotions.”

Prof. Martinez and colleagues photographed 130 female and 100 male volunteers who were asked to provide appropriate facial expressions to verbal cues such as “you just got some great unexpected news” or “you smell a bad odor.”

The facial muscles used prominently in each of the 5,000 resulting photographs were then tagged according to the Facial Action Coding System (FACS) created by psychologist Paul Ekman. Using the FACS data, the researchers were able to run searches cross-referencing similarities and differences in expressions.

From these distinct expressions, Prof. Martinez and team recorded a total of 21 emotions. Many of these were expressed by participants in an almost universal way – 99% of the volunteers depicted happiness by stretching their mouths into a smile, for example.

The “compound emotions” were also generally expressed using the same facial expressions. About 93% of the participants made the same face for “happily surprised,” which involved opening their eyes wide and raising their cheeks in a combination of a smile and a surprised face.

Even more contradictory compound emotions, such as “happily disgusted” – defined by Prof. Martinez as “how you feel when you watch one of those funny ‘gross-out’ movies and something happens that’s really disgusting, but you just have to laugh because it’s so incredibly funny” – had a universal expression. In this case: scrunched up eyes and nose, but with a smile.

Prof. Martinez believes this cognitive research could have some therapeutic applications. For instance, he explains, the model may be used in treatment of conditions like autism or post-traumatic stress disorder (PTSD):

For example, if in PTSD people are more attuned to anger and fear, can we speculate that they will be tuned to all the compound emotions that involve anger or fear, and perhaps be super-tuned to something like ‘angrily fearful’? What are the pathways, the chemicals in the brain that activate those emotions? We can make more hypotheses now and test them. Then, eventually, we can begin to understand these disorders much better and develop therapies or medicine to alleviate them.”

Recently, Medical News Today reported on research into a computer model that could predict with 85% accuracy whether people were lying or not based on their facial expressions.