Tailoring Lexical Choice to the User’s Vocabulary in Multimedia Explanation Generation

TAILORING LEXICAL CHOICE TO THE USER'S VOCABULARY IN MULTIMEDIA EXPLANATION GENERATION Kathleen McKeown Jacques Robin Michael Tanenblatt Department of Computer Science 450 Computer Science Building Columbia University New York, N.Y. 10027 { kathy,robin,tanenbla} @cs.columbia.edu ABSTRACT In this paper, we discuss the different strategies used in COMET (COordinated Multimedia Explanation Testbed) for selecting words with which the user is familiar. When pictures cannot be used to disambiguate a word or phrase, COMET has four strategies for avoiding unknown words. We give examples for each of these strategies and show how they are implemented in COMET. 1. Introduction A language generation system should select words that its user knows. While this would seem to involve simply selecting a known word instead of an un- known word (as is done, for example, in [1]), in many cases it requires entirely rephrasing the rest of the sentence. For example, in our domain of equipment maintenance and repair, if the user does not know the word "polarity," a sentence like "Check the polarity." will be rephrased as "Make sure the plus on the batte~,lines up with the plus on the battery compartment. Even when alternative words can be used-instead of an unknown word (e.g., a descriptive expression can be used instead of an object name), the alternative phrase may interact with other parts of the sentence which then need to be reworded as well. In this paper, we discuss the different strategies used in COMET for selecting words with which the user is familiar. Since COMET integrates text and pictures in a single explanation 1, unknown words are fre- quently disambiguated through accompan, ying. pic- tures. For example, when the accompanying picture clearly shows the object and its location, COMET will use the most common object name even if the user is unfamiliar with the name 2. When pictures can- not be used to disambiguate a word-or phrase, COMET has four strategms for avoiding unknown words: 1. Selecting an alternative word or phrase (e.g., generating "some number" in- stead of "arbitrary number' ') 2. Rephrasing by providing conceptual definitions (e.g., generating "Make sure the plus on the battery lines up with the plus on the battery compartment." in- stead of "Check the polarity") 3. Rephrasing by generating descriptive referring expressions (e.g., generating "the cable that runs to the KY57" in- stead of "the COMSEC cable' ') 4. Using past discourse to construct a referring expression (e.g., generating "Test the cable you just removed." in- stead of "Test the COMSEC cable." if the user had previously been instructed to remove this cable.) In the following sections, we first t?rov!de an over- view of lexical choice in COMET, snowing how and where it occurs in the overall system. Each of the strategies is then described in turn, prefaced by a brief discussion of disambiguation of unknown terms through pictures. Finally, we compare our work with previous work in the area. 1See [2] for a system overview and [3, 4] for details on media coordination in COMET. 2This …

Content truncated to 500 words