VUI Study Finds Simpler Is Better

NEW YORK (SpeechTEK 2008) -- Spit it out and make it simple. That’s what SpeechCycle’s Ethan Levine and David Suendermann found in a recent voice user interface study, the results of which they presented today at SpeechTEK 2008 in New York’s Marriott Marquis.

Levine and Suendermann looked at a cable TV troubleshooting application that took millions of calls per month. They started the study to identify problems with the current application. They found that call completion rates after the initial problem capture was more than twice the overall completion rate, but that too many callers were falling out of the menu and not giving the reasons for their calls.

The current baseline dialogue in question started like this: To make sure we do the right thing, please select one of these six main trouble areas. When I’m done listing them, you can say 'repeat' to hear them again. So please choose one of these...

The six trouble areas were then listed, followed by another request for a choice and the option to repeat the menu again.

"That was basically like War and Peace for our callers," Levine said. "Most people just chose the last thing they heard."

They found that some of the re-prompts were flawed and there was no touchtone fallback option. "If the caller didn’t get it right the first time, we didn’t offer them anything additional," Levine said.

So, they came up with two different approaches to address the problem.

For Alternative 1, they removed the lengthy introduction and followed it with two menus. The first menu had fewer, clearer options; the second menu was a fallback for lower-volume problems. In the two-menu structure, the caller would only hear certain options if he said,  Help me with something else.

Alternative 2 was similar, but had all of the same options as Alternative 1 in a single menu approach.

What they found was that with Alternative 1 people were more likely to say, Help me with something else, even if the option they wanted was listed.

<!--[if !supportEmptyParas]--><!--[endif]-->"Alternative 2 worked the best," Suendermann said, "because people were better able to explain what their choices were in terms of the representation of the choices in the prompt. Finally, looking at the outcome of the call, we see once again that Alternative 2 has a much higher chance of getting automated than the baseline as well as Alternative 1."

In getting back to their initial motivation for the study, Levine and Suendermann wanted to see if their redesign helped by showing what the problem actually was.

They found three things. First, callers could experience elevated frustration when they encounter a split-menu approach. Second, capturing the problem does not directly relate to higher call completion, and third, brevity of a dialogue has a small positive effect on problem capture but a significant positive effect on customer experience indicators.

"We saw a clear positive effect on caller experience," Suendermann said. "Alternative 2 is the winner of this competition, meaning that you should have more choices in one menu instead of having a two-level approach."

In fielding questions from the audience, Levine and Suendermann revealed that they looked at 9,000 to 10,000 calls for each variation. They had a call back option available, but did not present it in the menu.