Avoid the "Old-Folks' Home" with a Speech-Enabled House
Joe is 81 years old and lives by himself in a small house in rural Minnesota. His children wanted him to move to an assisted-living facility when his wife died two years ago, but he steadfastly overruled their suggestion. As a compromise, his children outfitted his entire house with a collection of electronic devices designed to help seniors and others with mild to medium dementia live independently.
The radio alarm wakes Joe at 7 a.m. A speech synthesis system on his PC converts a list of the day’s scheduled activities into a voice that Joe hears through his Bluetooth-enabled hearing aid. Earlier, Joe’s daughter, Randi, sent an email to inform the activity scheduler in Joe’s PC that she and her two daughters would be stopping by to see him in the afternoon. Joe is delighted to be reminded of the visit by his two favorite granddaughters.
Joe goes about his usual morning activities, but forgets to take his medicine. An unmoved medicine bottle reports its inactivity to the PC, which creates a message delivered through Joe’s hearing aid: Remember to take your medicine.
After taking his medicine, Joe prepares breakfast; he fries bacon and two eggs. Joe uses the last two eggs from his refrigerator, so he records the words buy eggs onto a verbal to-do list by speaking into a microphone embedded into the frame of his glasses. Joe will replay the verbal to-do list when he goes shopping with his daughter later in the day.
While eating breakfast, Joe hears a message via his hearing aid that originates from the stove reminding him to turn off the flame if he is finished cooking.
In the hallway are pictures of Joe’s children, grandchildren, and friends, as well as a picture of his late wife. A small camera in Joe’s glasses captures the barcode included in her photograph and sends a message via Joe’s hearing aid asking if he would like to listen to a recording that his wife made before she died.
Talking Pictures
After listening to the recording, Joe looks at the barcoded pictures of his four children. As he gazes at the picture of Randi, he hears the message: Randi will visit you this afternoon at 2 p.m. Joe speaks the word telephone into the microphone embedded in his glasses, and in a few seconds he is connected to Randi. Joe hears Randi’s words via his hearing aid, and his speech is captured by the microphone embedded in his glasses. Randi and Joe discuss the afternoon’s visit for about five minutes.
Joe then looks at a photo of his best friend, Sam, and is connected to him by phone. After chatting for a few minutes, they decide to play a round of virtual video golf. A camera analyzes Joe’s swing and calculates the trajectory of the golf ball. Both Joe and Sam can see the ball miss the hole by only a few inches.
Later in the day, Joe is awakened from his nap by the words spoken to his hearing aid: It is time for lunch. Joe prepares lunch as he excitedly waits for Randi and his granddaughters to arrive. When they finally arrive, one granddaughter bursts out, “Hello, Grandpa.” A camera captures her image, while a microphone captures her voice. The PC applies vision and speech algorithms to quickly determine that the girl’s name is Susie, which is spoken to Joe via his hearing aid. Joe gives her a big hug and a powerful, “How is my favorite granddaughter, Susie?” The other granddaughter exclaims, “What about me?” The PC’s recognition software whispers Anna’s name. “How is my other favorite granddaughter, Anna?” Joe asks, giving her a big hug as well.
Susie and Anna decide they want to watch TV. They adjust the volume for their own comfort. The TV audio is routed to Joe’s hearing aid, which adjusts the sound level so that Joe can also hear it.
Randi asks Joe how the new electronic devices are working. Joe lies: “I don’t know. They’re still in the box. Maybe I’ll take them out and try them when I get older.”
Joe and his family don’t really exist, but there are thousands of Joes who resist moving to assisted-living facilities. With assistive devices, it may be possible to extend their days in their familiar surroundings. All of the technology described above exists, but most of it is not available yet in a form for the public that can be easily integrated and used. This seems like a big business opportunity, especially as the Baby Boomers reach retirement age.
James Larson, Ph.D., is co-chair of the World Wide Web Consortium’s Voice Browser Working Group and author of the home-study guide The VoiceXML Guide. He can be reached at jim@larson-tech.com.