Tauira: A tool for acquiring unknown words in a dialogue context

Tauira: A tool for acquiring unknown words in a dialogue context Maarten van Schagen Human Media Interaction Group University of Twente 7500AE Enschede the Netherlands schagen@cs.utwente.nl Alistair Knott Department of Computer Science University of Otago Dunedin 9015 New Zealand alik@cs.otago.ac.nz Abstract This paper describes a tool for acquiring un- known words, which operates in a bilingual human-machine dialogue system. When the user’s utterance includes a word which is not in the system’s lexicon, the system initiates a subdialogue to find out about the new word, by querying the user about the syntactic validity of a number of example sentences generated au- tomatically from the grammar’s test suite. The tool can handle multiple unknown words, reg- ular morphology and translation of new words within a very complex unification feature struc- ture type hierarchy. 1 Introduction A major problem for current wide-coverage symbolic parsing systems is the existence of un- known words: words which appear in the in- put strings to be parsed, but which are not in the system’s lexicon. To give a representative example: when the English Resource Grammar (ERG: Copestake and Flickinger (2000)) was tested over a portion of the British National Corpus, unknown words accounted for 40% of all unparsed sentences (Baldwin et al., 2004). There are several ways of tackling this prob- lem. Some of these are offline, and simply in- volve making the lexicon bigger, adding new en- tries by hand or by some semiautomatic process. Other methods are on-line, and are designed to cope with unknown words which are encoun- tered during the parsing process itself. This paper is concerned with on-line meth- ods for dealing with unknown words. We fo- cus on methods which are appropriate for im- plementation in a human-machine dialogue system. In this application, a human user en- ters a sentence; the system then parses the sen- tence, derives a semantic interpretation, and generates a suitable response. If the user’s sen- tence contains an unknown word, then the sys- tem must somehow compute information about this word’s syntactic characteristics (so that the sentence can be parsed), and about its seman- tics (so that the sentence can receive an inter- pretation). Some of this information can typi- cally be inferred from the context in which the new word appears—but it is unusual if it can all be provided this way. Typically, some other source of knowledge needs to be actively con- sulted before the dialogue can resume. A dialogue application opens up some new possibilities for unknown word processing, be- cause when the system encounters an unknown word, it can initiate a subdialogue to find out more about this word. We will begin in Section 2 by outlining current approaches to unknown words in parsing, both in dialogue and non-dialogue applications. In Section 3 we will propose a new approach, which combines and extends existing approaches. In Section 4 we will present a new tool for unknown word processing which implements the proposed ap- proach. Section 5 gives some examples of word- …

Content truncated to 500 words