Defense Agency Concludes Field Tests for Auto-Translators

GAITHERSBURG, Md. -- Researchers at the National Institute of Standards and Technology (NIST) Friday wrapped up field-testing of prototype, real-time, two-way translation systems for the Defense Advance Research Projects Agency (DARPA), concluding that the lack of a genuine two-way translation device inhibits the ability of American soldiers to secure critical information and communicate with the local population.      

"Effective two-way translation devices would represent a major advance in field translators," Craig Schlenoff, project leader of the NIST evaluation project, said in a press release.  Phrase-based translators, according to Schlenoff translate English into pre-recorded Arabic phases and can’t translate Arabic into English at all.

The DARPA program, called TRANSTAC (Spoken Language Communication and Translation System for Tactical Use), currently focuses on English and Iraqi Arabic. NIST ran a series of laboratory and outdoor evaluation tests on prototype systems with English-speaking U.S. Marines and Iraqi Arabic speakers at its Gaithersburg, Md., campus July 16-20. In each of the exercises, NIST measured system capabilities in speech recognition, machine translation, noise robustness, user interface design, and performance on limited hardware platforms. Six systems were involved in the testing, including IBM, SRI, BBN, Fluential, Carnegie Melon University, and the University of Southern California.  

These tests, held at six-month intervals, are now entering their second year. "DARPA has different phases of the TRANSTAC program and each phase lasts a year," Schlenoff explains. "What they want is to get two points: they want to know how they did at the end of each year, and an interim point regarding how much the system has improved."  He adds: "Every year, they look to have a 100 percent greater probability that what’s being said is being translated properly. They did indeed meet that when they moved from Phase 1 to Phase 2. We haven’t analyzed the data yet, but from a visual perspective it looks as if it’s coming along nicely. DARPA wants to make sure the technology is advancing before they throw more money into the program."

According to Schlenoff, there are three main concerns: automatic speech recognition, machine translation, and text-to-speech. And each component offers its own challenges.  "I can’t say that any one works flawlessly or that any one is the biggest hurdle," says Schlenoff.  "Everything is improving, but still has room to improve."

NIST’s role is to process the data gleaned from the tests and turn it over to DARPA, showing the development of the TRANSTAC systems over time.  This allows DARPA to make educated decisions regarding which specific technologies to pursue.

Ultimately, DARPA hopes to gain the capabilities to develop an automatic translator system in a new language within 90 days of receiving a request for that language.  NIST has yet to finish analyzing the data. However, from first glance, Schlenoff says that the biggest challenge in accomplishing this goal "is not so much the time but the quantity of data you feed into the systems, which works on a statistical model. The more data you chug into it, the better it’s going to do."