General Competition (November 1993)

Advanced Spoken Language User Interfaces for Computer Applications

Sponsor: Scansoft (formerly Kurzweil Applied Intelligence, Div. of Lernout & Hauspie)

• Project Performance Period: 3/1/1994 - 2/28/1997
• Total project (est.): $2,398,650.00
• Requested ATP funds: $1,776,750.00

Although fictional characters have been able to talk naturally to their computers for some time (and get results), in the real world speech-recognition interfaces for personal computers are quite rudimentary. For actually controlling a computer program (as opposed to data input), the user generally has been restricted to the syntax of standard PC software commands, which are not always obvious. Speech recognition would be a much more valuable and important tool if users could interact with programs in their natural language. The catch is that true natural-language interpretation is a very large -- perhaps unsolvable -- problem. Kurzweil AI proposes to attack the problem by developing a specialized, limited subset of English which is simple enough to allow the computer to interpret the commands, yet powerful enough to allow the user to give relatively straight-forward "natural" commands without consulting a manual. Key issues, for example, are how to deal with anaphora and deixis: sentences like "Capitalize the first word and put it in boldface." and "Italicize these words." (pointing to several words with a mouse.) In addition to developing a simplified syntax and semantics appropriate to controlling typical computer programs, the proposed "Talk, Touch, Type" (T3) interface will require advances in the hardware and software for continuous speech recognition, the recognition and interpretation of continuous streams of words. Many existing speech-recognition systems rely on receiving discrete, individual words, a much simpler task. Other issues include the software architecture and interface to the application program and a number of human factors essential to a useful computer interface. As a test case for this project, KAI proposes to develop a natural-language interface for a popular word processor.

For project information:

Francis Ganong, (781) 684-4812

ATP Project Manager

Barbara Cuthill, (301) 975-3273
barbara.cuthill@nist.gov