Natural Language in Four Spatial Interfaces

Natural Language in Four Spatial Interfaces Kenneth Wauchope, Stephanie Everett, Dennis Perzanowski, Elaine Marsh Navy Center for Applied Research in Artificial Intelligence Naval Research Laboratory, Code 5512 4555 Overlook Avenue, S.W. Washington, DC 20375-5337, USA [wauchope ] severett ] dennisp I marsh] ~aic. nrl. navy. mil Abstract We describe our experiences building spo- ken language interfaces to four demon- stration applications all involving 2- or 3-D spatial displays or gestural interac- tions: an air combat command and control simulation, an immersive VR tactical sce- nario viewer, a map-based air strike sim- ulation tool with cartographic database, and a speech/gesture controller for mobile robots. 1 Introduction The NAUTILUS natural language processor has been under development at our facility since about 1988. During those years it has been used and tested in five different demonstration projects, four of which we describe in some detail in this report: an air combat command and control simulation, an im- mersive VR tactical scenario viewer, a map-based air strike simulation tool with cartographic database, and a speech/gesture controller for mobile robots. All four applications involve spatial displays or in- teractions, including 2D map-based graphical dis- plays (radar screen, geographic map), 3D perspec- tive scenes, and hand gesture input. 2 NAUTILUS NAUTILUS is built around an early version of the PROTEUS chart parser from New York University's Courant Institute (Grishman, 1986). The three sub- sequent system components were developed at our own facility. 2.1 PROTEUS PROTEUS syntactic grammars consist primarily of two rule types: context-free rules written in BNF no- tation, and restriction rules written in a high-level 8 algorithmic language. Each restriction rule is at- tached to a particular nonterminal in the right-hand side of one or more context-free rules. During pars- ing the restriction fires immediately after that non- terminal has been constructed, testing the subtree at that point for well-formedness, or attaching an attribute value for use later on. Nodes rejected by a restriction are not added to the active chart and so do not contribute to the remainder of the parse. 2.2 TINSEL One of the attributes composed by PROTEUS dur- ing parsing is an operator-operand regularized form intended to serve as the representation to which semantic selection and interpretation rules can be applied. The TINSEL semantic interpreter (Wau- chope, 1990) applies case-frame rules and selection restrictions to the PROTEUS regularized output. The interpreter can either he invoked post-parse (ap- plied top-down to each candidate sentential regu- larization) or interleaved with PROTEUS, testing each individual clausal or noun phrase immediately upon construction. In interleaved mode, if a node's regularization does not pass the case-frame or se- lection criteria then the node is not added to the chart, which can prune the search space and reduce parsing time considerably. If the node does pass se- lection, its regularization is augmented with the rel- evant semantic class and role information, becoming an intermediate semantic representation suitable for further processing such as reference resolution and quantifier scoping. The TINSEL interpreter is primarily model- driven, which is …

Content truncated to 500 words