Mondly Launches VR and Voice-Based Language Tool
ATi Studios, creators of Mondly, has released a virtual reality app for language education that combines the artificial intelligence technology behind chatbots with speech recognition in virtual reality.
Learn Languages VR by Mondly allows people to experience life-like conversations with virtual characters, in 28 languages, from the comfort of home. The app offers instant feedback on pronunciation, suggestions that enrich learner's vocabulary, and surprises that transform learning a language into a fun experience.
Mondly's conversational voice chatbot for language learning was first released in August 2016. The technology uses advanced speech recognition that understands millions of phrases and words in 28 languages. Since its release, it has been used by millions of people worldwide and has learned millions of new phrases and responses, thanks to the chatbot's self-learning technology.
To bridge the technologies behind its chatbot, speech recognition, and VR, the company had to develop a new automatic voice detection system.
"The toughest challenge was to make the speech interaction feel natural in VR. Because VR is so immersive, we quickly realized that traditional speech interaction models that require a tap or a voice command to enable speech recognition would kill the flow of the experience," said Alexandru Iliescu, CEO and co-founder of ATi Studios, in a statement. "So we developed our own automatic voice detection system. It calibrates to the room's background noise, and it ignores unintentional sounds. The result is amazing. The conversations with the virtual characters happen as naturally as real-life conversations; they just flow."
Learn Languages VR by Mondly allows users to practice conversing in a new language with virtual characters; all within realistic situations that teach language fundamentals and help build their confidence.
The first version offers three scenes to explore:
- The Restaurant-Practice ordering food and drinks in a small bistro.
- The Eurotrain-Have a conversation and make new friends on a train in Europe.
- The Hotel-Check into a boutique hotel and score a room with a view.
"Fluency in a language is not how many words you know, but how well you communicate with the words you do," Iliescu said. "Our goal with Mondly is to push the envelope of language learning with new technologies that drive our users to become better communicators and have richer life experiences. And the Oculus team and the virtual reality technology they've created offered us the perfect environment for achieving this goal."
The app currently supports U.S. and U.K. English, German, French, Italian, Spanish, Russian, Romanian, Greek, Portuguese, Arabic, Croatian, Czech, Danish, Dutch, Finish, Hebrew, Hindi, Hungarian, Japanese, Korean, Norwegian, Polish, Swedish, Turkish, Ukrainian, Vietnamese, and Thai.