Opening the Kimono
I was recently pulled into discussions with several organizations for whom speech did not meet expectations. In each case, the organization’s leaders threw up their hands, cried “uncle,” and went back to their comfortable touch-tone applications. Now, some folks are passionate about speech technology, so the mere inkling that someone might be less than a fan seems hard to fathom. On the other hand, keeping a circumspect, decidedly agnostic view of technology is what human factors folks do best. It’s easy to emotionally react in a so-called “failure” situation and conclude that speech technology is useless—a conclusion of convenience that’s not necessarily accurate.
For our New Year’s resolutions, we speech types need to be a little more reflective and honest about the gating factors on telephone-based interaction success (speech and touch-tone). Designers can support success by knowing and helping to mitigate the pitfalls that make speech unsuccessful.
If we peek inside the kimono, we will see that each organization where speech failed had some combination of the following characteristics:
• Decentralized control of the interactive voice response (IVR) system—This didn’t necessarily mean the company didn’t have an IVR team. In fact, several had such teams. On paper, the team was in control of the IVR, but, in reality, it had no final decision-making authority and was beholden to multiple stakeholder groups who could propose changes at will. After script adjustments, the resulting user experiences had become so convoluted they were unusable by the average consumer.
• Lack of knowledge diversity—Many teams lacked at least one individual with deep knowledge of the behavioral sciences and communication. This lack of an advocate who can articulate user needs, apply human behavior, prioritize conflicting requirements, and translate it all into an efficient prompt often leads to unusability.
• Limited outcome metrics—This problem had three flavors: not measuring outcomes at all, having metrics but not analyzing diagnostic data, or not identifying appropriate actionable outcomes of data analysis. Regardless of the cause, data that was not locked to the dialogue design led to disaster.
• Lack of trust—This problem manifested itself in two ways: Either a single vendor provided consultancy support, but its ideas were rejected or modified, usually due to technical limitations, business processes, or historical “this-is-the-way-we’ve-always-done-it” attitudes; or several vendors, often with competing ideas, were involved in the project, allowing the organization to pick and choose recommendations and often rendering each vendor’s advice ineffective.
• Lack of predeployment usability testing—One of the biggest advantages of usability testing is its predictive power. If a test is well-designed and the participant sample matches the user population, then you get a pretty accurate idea of what to expect when an application deploys. In many cases, usability testing wasn’t done, or it wasn’t done well, which made production usage a giant (and unwelcome) surprise.
• Spaghetti-like business logic—Although this is a very common problem, its implications are clear. If the script is hamstrung by business logic that requires a historical overview and organizational chart to explain, the user won’t be able to navigate the application.
• Lack of change management—This problem came in two versions: design, deploy, and put it in the closet and forget about it; or let anyone with an opinion propose and implement changes. In many companies, changes were proposed for unverified problems, introducing more serious usability problems than what originally existed. From there, it was a slippery slide to unusability.
• Ripple-effect refusal—When executed well, a new user experience often has broad effects on a business. It could mean changes in agent queuing, training, and skill sets, changes in internal processes and business logic, or new linkages between back-end databases that never talked to each other before. It could require more rigorous data capture plus ongoing reporting and analysis. If a company shies away from these changes or simply refuses them, then it essentially puts a straightjacket on the IVR script.
What’s to be gained from opening this kimono? First, vendors need to work on relationship-building, particularly trust, in dealing with clients. Introducing transforming skill sets and perspectives to an organization is a delicate matter, particularly for companies firmly entrenched in what they do and their strong sense of history. For organizations, honestly evaluating internal processes and attitudes before embarking on a speech application project is key. In some cases, critical evaluation could result in a decision to slow or postpone the introduction of speech technology in preference of foundational projects that will pave the way for success with speech later. Kimono closed.
Melanie Polkosky, Ph.D., is a social-cognitive psychologist and speech language pathologist who has researched and designed speech, graphic, and multimedia user experiences for more than 12 years. She is currently a human factors psychologist and senior consultant at IBM. She can be reached at polkosky@comcast.com.