‘Emotional’ Computers

High-tech tutors encourage, empathize with math students

‘Emotional’ Computers
Photograph by Bloom Image/ Getty Images

“Emotional computer” sounds like an oxymoron. “I’m a computer scientist, and I know the computer is not emotional in any sense,” says Beverly Woolf. But she is one of a few pioneering researchers developing just that: a computer that can identify its user’s mood and respond—with encouragement, empathy, even advice. Using sensors and cameras, this technology determines an individual’s emotional state from indicators such as heart rate and facial expression, with up to 80 per cent accuracy.

So far, emotional computers have been used by one population especially vulnerable to frustration, distraction and misery: students learning math. Woolf, a professor at the University of Massachusetts Amherst, and her colleagues have created a computer system that gauges the skills and feelings of students as they perform math online. The computer then tailors its questions and voice messages—offering, say, congratulations or help. The goal is to give each student the cognitive and emotional support he or she needs for optimal learning—but can’t always get from a teacher whose attention is divided. Woolf calls this the “personalization of education,” and says “every child can learn. We just have to figure out how to teach them.”

This is a major advancement in the realm of “intelligent tutors” or computer-based learning tools that focus on cognitive ability alone. “It is the frontier of knowledge,” says Claude Frasson, a world-renowned trailblazer in this area of science, and director of the elite Higher Educational Research on Tutoring Systems lab at the Université de Montréal. “This is a convergence of artificial intelligence, educational psychology and neurology.” There is a growing appreciation among computer scientists of the impact that feelings have on knowledge acquisition: “A lot of people now are starting to say we have to worry about the emotion,” says Woolf, who is spending her sabbatical at the Université de Montréal and the Université du Québec à Montréal. “People don’t learn if they’re in a sad state or anxious.”

Walk into a classroom in Arizona or Massachusetts this spring and you may find students working on emotional computers as part of Woolf’s latest round of studies, supported by the U.S. National Science Foundation. Using each school’s own computers, the researchers plug algorithms into each Web camera so it captures eyebrow and mouth movement, which are indicators of sadness, boredom or interest. The camera also identifies whether the student’s head is turned left or right, rather than focused on the math exercises in front of him or her.

As well, pillows containing sensors are placed on the seat and back of the student’s chair. These detect shifting or fidgeting, and whether a student is leaning forward, which suggests engagement. The conventional mouse is replaced with one that features six internal sensors that measure the student’s grip. Usually, the more trouble he or she is having, the harder the mouse is squeezed. Finally, each student gets a wrist bracelet that identifies activity levels and alertness through factors such as increased blood circulation. (The researchers eventually hope to use only the camera to get emotional information.)

Once all of the components are set up, the students begin an animated online math program. They choose a “learning companion”—male or female, Caucasian, African-American or Hispanic. Woolf’s team, which is co-led by Ivon Arroyo, also a professor at Massachusetts University, is assessing if students prefer to learn from someone of their own gender and race. To foster a sense of adventure, the learning companions and problem-solving questions are based on the work of real biologists studying orangutans in Borneo.

So far, the research has turned up fascinating data. When students feel bad, they don’t want their learning companion to reflect that emotion—by looking sad, angry or sleepy. This is surprising because mirroring usually makes people comfortable. “So one needs to be very positive,” says Woolf. What’s more, female students like more encouraging feedback from learning companions than boys, who mute the characters twice as often. Females may be more “emotionally tuned in” than males, explains Woolf, because the part of the brain related to social recognition is fully formed sooner. Furthermore, low-achieving students also exhibit more enthusiasm for math after these tutorials.

It’s unclear yet how emotional computers may impact grades, but previous research by Woolf has shown computer-based tutors boost math scores by up to 20 per cent. The emotional support, she believes, enhances the learning experience: “Students say, ‘I like math better, I’m less frustrated.’ So that is definitely an improvement.”