The Efficiency of Multimodal Interaction for a Map-based Task

The Efficiency of Multimodal Interaction for a Map-based Task Philip COHEN, David McGEE, Josh CLOW Center for Human-Computer Communication Oregon Graduate Institute of Science & Technology 20000 N.W. Walker Road Beaverton, Oregon 97006 { pcohen, dmcgee } @cse.ogi.edu Abstract This paper compares the efficiency of using a standard direct-manipulation graphical user interface (GUI) with that of using the QuickSet pen/voice multimodal interface for supporting a military task. In this task, a user places military units and control measures (e.g., various types of lines, obstacles, objectives) on a map. Four military personnel designed and entered their own simulation scenarios via both interfaces. Analyses revealed that the multimodal interface led to an average 3.5-fold speed improvement in the average entity creation time, including all error handling. The mean time to repair errors also was 4.3 times faster when interacting multimodally. Finally, all subjects reported a strong preference for multimodal interaction. These results indicate a substantial efficiency advantage for multimodal over GUI-based interaction during map-based tasks. Introduction Nearly two decades ago at ACL'80, Professor Ben Shneiderman challenged the field of natural language processing as follows: In constructing computer systems which mimic rather than serve people, the developer may miss opportunities for applying the unique and powerful features of a computer: extreme speed, capacity to repeat tedious operations accurately, virtually unlimited storage for data, and distinctive input/output devices. Although the slow rate of human speech makes menu selection impractical, high-speed computer displays make menu selection an appealing alternative. Joysticks, light pens or the "mouse" are extremely rapid and accurate ways of selecting and moving graphic symbols or text on a display screen. Taking advantage of these and other computer-specific techniques will enable designers to create powerful tools without natural language commands. [20, p. 139] He also challenged us to go beyond mere claims, but to demonstrate the benefits of natural language processing technologies empirically. Since then, not only has there been a long period of unprecedented innovation in hardware, software architectures, speech processing, and natural language processing, but NLP research has also embraced empirical methods as one of its foundations. Still, we have yet to defend claims empirically that technologies for processing natural human communication are more efficient, effective, and/or preferred, than interfaces that are best viewed as "tools," especially interfaces involving a direct manipulation style of interaction. The present research attempts to take a small step in this direction. In fact, it has often been claimed that spoken language-based human-computer interaction will not only be more natural but also more efficient than keyboard-based interaction. Many of these claims derive from early modality comparison studies [1], which found a 2-3 fold speedup in task performance when people communicated with each other by telephone vs. by keyboard. Studies of the use of some of the initial commercial speech recognition systems have reported efficiency gains of approximately 20% - 40% on a variety 331 of interactive hands-busy tasks [10] compared with keyboard input. Although these results were promising, once the time needed for error correction was …

Content truncated to 500 words