The Multimedia Articulation of Answers in a Natural Language Database Query System

The Multimedia Articulation of Answers in a Natural Language Database Query System Susan E. Brennan Stanford University and Hewlett Packard Labs 1501 Page Mill Road Palo Alto, CA 94304 Abstract This paper describes a domain independent strategy for the multimedia articulation of answers elicited by a natural language interface to database query ap- plications. Multimedia answers include videodisc im- ages and heuristically-produced complete sentences in text or text-to-speech form. Deictic reference and feedback about the discourse are enabled. The inter- face thus presents the application as cooperative and conversational. 1 Introduction It is useful to evaluate human-computer communica- tion in light of Grice's cooperative principle and max- ims [Gri75]. Recently there has been much interest in a "cooperative response" paradigm for interfaces to database query and expert systems [Ste87]. The most promising strategies in this area of investigation involve applying insights gained from psycholinguis- tics research in order to create better conversational human/computer interfaces. However, inventing ad- equate user modeling and inferencing systems for this purpose is no easy task, and much of the literature on the subject describes proposals for systems yet unimplemented or theoretical approaches which may depend heavily on a particular domain model. Our multimedia articulator consists of principled solutions which have been implemented in a domain indepen- dent manner and which produce answers that are rea- sonably relevant, informative, and conversational in style. Such a system makes it possible to begin to study users interacting with a question-answering ap- plication. 2 System overview The system described here functions as a conversa- tional human/computer interface to database query systems. It consists of a natural language front end and a module which articulates multimedia answers. The system accepts well-formed strings as input; these sentences are interpreted by an HPSG-based parser [PS87] which produces a parse tree. After fur- ther processing by a semantics module, a pragmatics processor [BFP87] and a disambiguator, a logical for- mula in the language NFLT [CP85] is produced. This formula is transduced into a database query. Two database query formats are currently supported: a frame-based representation language, HPRL, and the standard relational database query language, SQL. Answers returned from the database are then pack- aged appropriately by the articulator for presentation to the user. The two database applications currently supported are a database of people and equipment (a subset of which we have proposed as a natural language evaluation test suite [FNSW87]), and a database of paintings by 19th century Dutch artist Vincent Van Gogh and his contemporaries. The latter database was based on the index to a commercially available videodisc [Nim82] and augmented from other sources. Both applications can be run on workstations config- ured with or without multimedia output devices. 3 Database answer format The driver of a database query application (i.e. the domain dependent part of the system) is responsible for returning answers in a list format which consists of a keyword specifying the type of the answer, followed by the answer itself. The answer types expected by the articulator are …

Content truncated to 500 words