Video: Benchmarking Voice Assistants, Pt. 6: Emotional IQ & Common Sense
Learn more about intelligent assistants at the next SpeechTEK conference.
Read the complete transcript of this clip:
Kathleen Walch: We know that humans have a better emotional IQ than machines, but in order for them to become truly intelligent, we need to start helping them learn how to understand emotions, especially if, depending on the situation that you're going to be using it in. Customer service, for example. So we asked questions such as should I send a congratulations notice for my friend's funeral, or when you yell, are you loud, and are friends people who like each other? So these are basic questions to understand relationships and dynamic and emotions.
Ronald Schmelzer: Right, and as Kathleen mentioned, like really the biggest application is in the customer service setting when you're trying to build chatbots or voice assistant bots that are supposed to operate autonomously, and you know, customer's irate or confused or overly enthusiastic.
Kathleen Walch: Or sad or stressed out. I mean, maybe they just got in a car accident for example.
Ronald Schmelzer: The interesting thing is like AI machine learning's actually being applied to pretty good use in sentiment analysis. This is sort of like your typical sentiment analysis thing, but also you want to sort of key off on words they use, so like for example, people saying I'm frustrated! Or like man, you asked me that question five times. These are sentiment analysis of not happiness, so let's see if these devices have been hopefully trained to key on these things.
- [Computer] Alexa, should I send a congratulations notice for my friend's funeral?
- [Alexa] Hmm, I'm not sure.
Ronald Schmelzer: That's a long pause. Do I have the cut on this one, sorry about that. Hello. I think I might have inserted the wrong, well.
- [Computer] Alexa, when you yell, are you loud?
- [Alexa] Sorry, I don't know that one.
Ronald Schmelzer: So, there's a lot of that. In that video, there's a lot of--
Kathleen Walch: You can see there's mostly reds in there.
Ronald Schmelzer: And Google is the one that gets. To the other side.
- Yeah, I moved the next one.
Ronald Schmelzer: Google is the one that basically gets the emotional IQ ones generally better than the other ones, but they're all kind of not the greatest at this, so this is kind of an area of continued work. We also decided to test if these devices had intuition and common sense, so we always joke that humans don't always have common sense, but. But in general, and Ron always says it's neither common nor sense. But in general, we wanted to see you know if these devices understood certain things. So are parents older than their children? Or should I tell a toddler to drive a bus? Is a thief a criminal?
Kathleen Walch: Right, and these are helpful when you're doing some things that are really helpful, and again, we're gonna bring in our little kitchen example here, so let's see if there's some, something that'll be more useful to us and relevant to what we're trying to do.
- [Computer] Alexa, should I drink milk if I'm lactose-intolerant?
- [Alexa] Sorry, I'm not sure about that.
- [Computer] Hey Google, should I drink milk if I'm lactose intolerant?
- [Google] On the website milk.procon.org, they say there is no reason for people with lactose intolerance to push themselves to drink milk. Indeed, milk and other dairy products do not offer any nutrients that cannot be found in a healthier form on other foods.
Ronald Schmelzer: Okay, kind of, it's kind of helpful. Make the decision yourself, human. That's what it's basically telling you to do.
Related Articles
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer present conclusions from their benchmarking tests on intelligent assistants in this clip from their presentation at SpeechTEK 2019.
27 Dec 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer discuss the seventh test in their benchmarking project, testing the intelligent assistants' understandings of slang and colloquialisms in this clip from their presentation at SpeechTEK 2019.
20 Dec 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer discuss the fourth test in their benchmarking project, testing the intelligent assistants' handling of reasoning and logic in this clip from their presentation at SpeechTEK 2019.
06 Dec 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer discuss the third test in their benchmarking project, testing the intelligent assistants' grasp of cause and effect in this clip from their presentation at SpeechTEK 2019.
29 Nov 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer discuss the first test in their benchmarking project, testing the intelligent assistants' handling of comparison questions in this clip from their presentation at SpeechTEK 2019.
22 Nov 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer discuss the first test in their benchmarking project, testing the intelligent assistants' grasp of basic concepts in this clip from their presentation at SpeechTEK 2019.
15 Nov 2019
Cognilytica Analysts Kathleen Walch & Ronald Schmelzer introduce the Cognilytica Voice Assistant Benchmark for testing the intelligence of devices such as Alexa, Siri, Google Home, and Cortana in this clip from their presentation at SpeechTEK 2019.
08 Nov 2019