FLUSH: A Flexible Lexicon Design

FLUSH: A Flexible Lexicon Design David J. Besemer and Paul S. Jacobs Artificial Intelligence Branch GE Corporate Research and Development Schenectady, NY 12301 USA Abstract Approaches to natural language processing that use a phrasal lexicon have the advantage of easily handling linguistic constructions that might otherwise be ex- tragrammatical. However, current phrasal lexicons are often too rigid: their phrasal entries fail to cover the more flexible constructions. FLUSH, for Flexible Lexicon Utilizing Specialized and Hierarchical knowl- edge, is a knowledge-based lexicon design that allows broad phrasal coverage. I. Introduction Natural language processing systems must use a broad range of lexical knowledge to account for the syntactic use and meaning of words and constructs. The problem of un- derstanding is compounded by the fact that language is full of nonproductive constructs--expressions whose mean- ing is not fully determined by examining their parts. To handle these constructs, some systems use a phrasal lex- icon [Becket, 1975, Wilensky and Arena, 1980b, Jacobs, .1985b, Steinacker and Buchberger, 1983, Dyer and Zernik, 1986], a dictionary designed to make the representation of these specialized constructs easier. The problem that phrasal lexicons have is that they are too rigid: the phrasal knowledge is entered in a way that makes it difficult to represent the many forms some expressions may take without treating each form as a dis- tinct "phrase". For example, expressions such as "send a message", "give a hug", "working directory", and "pick up" may be handled as specialized phrases, but this over- looks similar expressions such as "give a message", "get a kiss", "working area", and "take up". Specialized con- structs must be recognized, but much of their meaning as well as their flexible linguistic behavior may come from a more general level. A solution to this problem of rigidity is to have a hier- archy of linguistic constructions, with the most specialized phrases grouped in categories with other phrases that be- have similarly. The idea of a linguistic hierarchy is not novel, having roots in both linguistics [Lockwood, 1972, Halliday, 1978] and Artificial Intelligence [Sondheimer et al., 1984]. Incorporating phrasal knowledge into such a hierarchy was suggested in some AI work [Wilensky and Arena, 1980a], but the actual implementation of a hier- 186 archical phrasal lexicon requires substantial extensions to the phrasal representation of such work. The Flexible Lexicon Utilizing Specific and Hierar- chical knowledge (FLUSH) is one component in a suite of natural language processing tools being developed at the GE Research and Development Center to facilitate rapid assimilation of natural language processing technology to a wide variety of domains. FLUSH has characteristics of both traditional and phrasal lexicons, and the phrasal portion is partitioned into four classes of phrasal entries: • word sequences • lexical relations • linguistic relations • linguistic/conceptual relations FLUSH's mechanisms for dealing with these four classes of specialized phrases make use of both general and specific knowledge to support extensibility. FLUSH is the lexical component of a system called TRUMP (TRansportable Understanding Mechanism Pack- age) [Jacobs, 1986b], used …

Content truncated to 500 words