How Can Speech Technology Help?

COVID-19, unlike most other epidemics in our experience, has a place for amateurs and non-physicians to help. Remarkable improvements in technology made this possible.

Here’s one piece of technology that has moved in the space of about three years from obscure and expensive to commonplace and relatively cheap. Many of my friends own 3-D printers, expert users who design and print parts for their various projects, both new designs and repairs. When the Great Mask Shortage began in March 2020, they stepped up and began churning out masks.

Then came a wave of higher-tech applications, but before I get there, I want to thank the lower-tech and personal contributions by the denizens of my Chicago neighborhood. A charitable organization found funding to distribute thousands of meals each week to people suddenly thrown out of work. The local volunteer fast-response paramedics found themselves with their hands full, and citywide, Chicago’s Community Emergency Response Team provided 311 staffing to answer COVID-19-related calls. These people donated time and sweat and money to help their neighbors, and we all owe them, and people like them across the world, a debt of gratitude.

The high-tech projects of COVID-19 included, as one might expect, websites dedicated to various ventures: databases about the epidemic itself; dedicated communication channels between researchers; support sites for various new charitable groups. Other sites supported volunteer and commercial ventures that seem like obvious projects, but, in my opinion, only in retrospect. For example, I think we can all agree now that any project that offers grandparents (such as myself) a way to more easily read books to distant grandchildren is extremely welcome.

Still other projects have the usual ominous ring. “Contact tracing” via apps on phones is one example—whether voluntary or not, and in many places, not. Here in the United States, I’m not entirely sanguine about signing up Big Data companies to pore through their records to track tens of millions of citizens.

And then we come to speech technology. What can speech technology do to help?

If you’ve ever been to a call center, or for that matter any cubicle farm, you’ve no doubt noticed it’s not set up for social distancing (“social distancing” is a misnomer, by the way—it’s asocial distancing). This means workers will work from home; this means a great deal of work for system administrators. While in theory this means the flexibility of text-to-speech (TTS) and automatic speech recognition (ASR) could help, in practice I’ve yet to run across a business using TTS for calls, or one that made a significant switch to ASR to listen to callers. The reason is quite simple: If your underlying call center technology does not currently support a remote-based workforce, ASR and TTS will not make this more fundamental problem vanish.

But the main issue that came to my mind about speech technology and the pandemic is related to medicine. I’ve seen any number of web pages that let you determine, based on your symptoms, if you should be tested for COVID-19. I know there are chatbots (experimental ones, as far as I can tell) to analyze symptoms, and I fully expect to see dedicated kiosks, complete with ASR, that will take over the tedious task of screening visitors; expect some to be deployed later this year to at least some businesses. I’ll guess that airports will get them eventually as well. Having answered the COVID-19 screening questions any number of times, what I’d really like at this point is an ASR application for my phone that hears the questions from the kiosk and answers on my behalf. No healthcare system I’ve used over the past two months offers automated pre-visit screening over the phone.

But what really intrigues me is the possibility of phone-based applications that screen for COVID-19 based on biometrics. Set aside smart watches that monitor your temperature, heart rate, or even your EKG. What about a smartphone application that analyzes speech to determine if you’re ill with COVID-19? Research groups have conjectured that COVID-19, which inflames the respiratory system, would produce measurable changes in speech, changes these groups say they’ve detected.

Perhaps this work will pan out and perhaps not. Maybe vaccines will arrive before we know for sure. Regardless, even if this work does not help this time around, it lays the foundation for future work. Perhaps one day a rapid test for a new respiratory pathogen can be deployed, not by shipping throat swabs, but deploying phone-based applications—a worthy goal, one everyone in the speech technology field can be proud of. 

Moshe Yudkowsky, Ph.D., is the president of Disaggregate Consulting and author of The Pebble and the Avalanche: How Taking Things Apart Creates Revolutions. He can be reached at speech@pobox.com.