AI Emotion Recognition: How Machines Learn to Read Our Faces

In an interview, Björn Schuller, an engineer and researcher at TU Munich and Imperial College London, describes how AI systems recognize emotions through mathematical models. He explains that his team uses a three-dimensional model of emotions: arousal, valence, and dominance, which helps map discrete categories like the six basic emotions: anger, disgust, sadness, surprise, fear, and joy. Schuller notes that psychologists disagree on definitions, and his systems, which have been in development for over 25 years, focus on speech analysis. These systems use deep learning and training data where humans rate emotions, meaning machines learn from human judgments and cannot yet outperform humans, typically achieving 60-70% accuracy. Critics like Lisa Feldman Barrett from Northwestern University question the scientific foundation of such recognition systems. Despite this, the market is growing: last year, startups and companies like Amazon, Microsoft, and IBM generated over $40 billion globally in emotion recognition, with projections to more than triple by 2034. Applications include psychotherapy coaching, market research, and lie detection in security agencies. Schuller notes that large language models and multimodal models are slowly entering this market, but a study from the University of Bern showed they outperform humans in emotional intelligence tests. Additionally, a DAK Gesundheit and Hamburg University Medical Center survey found that one in ten teenagers prefers confiding in a chatbot over a friend.