Combining Multiple, Large-Scale Resources in a Reusable Lexicon for Natural Language Generation

Combining Multiple, Large-Scale Resources in a Reusable Lexicon for Natural Language Generation Hongyan Jing and Kathleen McKeown Department of Computer Science Columbia University New York, NY 10027, USA {hjing, kathy} @cs.columbia.edu Abstract A lexicon is an essential component in a gener- ation system but few efforts have been made to build a rich, large-scale lexicon and make it reusable for different generation applications. In this paper, we describe our work to build such a lexicon by combining multiple, heteroge- neous linguistic resources which have been de- veloped for other purposes. Novel transforma- tion and integration of resources is required to reuse them for generation. We also applied the lexicon to the lexical choice and realization com- ponent of a practical generation application by using a multi-level feedback architecture. The integration of the lexicon and the architecture is able to effectively improve the system para- phrasing power, minimize the chance of gram- matical errors, and simplify the development process substantially. 1 Introduction Every generation system needs a lexicon, and in almost every case, it is acquired anew. Few ef- forts in building a rich, large-scale, and reusable generation lexicon have been presented in liter- ature. Most generation systems are still sup- ported by a small system lexicon, with limited entries and hand-coded knowledge. Although such lexicons are reported to be sufficient for the specific domain in which a generation sys- tem works, there are some obvious deficiencies: (1) Hand-coding is time and labor intensive, and introduction of errors is likely. (2) Even though some knowledge, such as syntactic structures for a verb, is domain-independent, often it is re-encoded each time a new application is un- der development. (3) Hand-coding seriously re- stricts the scale and expressive power of gener- ation systems. As natural language generation is used in more ambitious applications, this sit- uation calls for an improvement. Generally, existing linguistic resources are not suitable to use for generation directly. First, most large-scale linguistic resources so far were built for language interpretation applications. They are indexed by words, whereas, an ideal generation lexicon should be indexed by the se- mantic concepts to be conveyed, because the in- put of a generation system is at semantic level and the processing during generation is based on semantic concepts, and because the mapping in the generation process is from concepts to words. Second, the knowledge needed for gen- eration exists in a number of different resources, with each resource containing a particular type of information; they can not currently be used simultaneously in a system. In this paper, we present work in building a rich, large-scale, and reusable lexicon for gener- ation by combining multiple, heterogeneous lin- guistic resources. The resulting lexicon contains syntactic, semantic, and lexical knowledge, in- dexed by senses of words as required by gener- ation, including: A complete list of syntactic subcategoriza- tions for each sense of a verb to support surface realization. A large variety of transitivity alternations for each sense of a verb to support para- phrasing. Frequency …

Content truncated to 500 words