Stochastic Language Generation Using WIDL-Expressions and its Application in Machine Translation and Summarization

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 1105–1112, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Stochastic Language Generation Using WIDL-expressions and its Application in Machine Translation and Summarization Radu Soricut Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 radu@isi.edu Daniel Marcu Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 marcu@isi.edu Abstract We propose WIDL-expressions as a flex- ible formalism that facilitates the integra- tion of a generic sentence realization sys- tem within end-to-end language process- ing applications. WIDL-expressions rep- resent compactly probability distributions over finite sets of candidate realizations, and have optimal algorithms for realiza- tion via interpolation with language model probability distributions. We show the ef- fectiveness of a WIDL-based NLG system in two sentence realization tasks: auto- matic translation and headline generation. 1 Introduction The Natural Language Generation (NLG) com- munity has produced over the years a consid- erable number of generic sentence realization systems: Penman (Matthiessen and Bateman, 1991), FUF (Elhadad, 1991), Nitrogen (Knight and Hatzivassiloglou, 1995), Fergus (Bangalore and Rambow, 2000), HALogen (Langkilde-Geary, 2002), Amalgam (Corston-Oliver et al., 2002), etc. However, when it comes to end-to-end, text-to- text applications – Machine Translation, Summa- rization, Question Answering – these generic sys- tems either cannot be employed, or, in instances where they can be, the results are significantly below that of state-of-the-art, application-specific systems (Hajic et al., 2002; Habash, 2003). We believe two reasons explain this state of affairs. First, these generic NLG systems use input rep- resentation languages with complex syntax and se- mantics. These languages involve deep, semantic- based subject-verb or verb-object relations (such as ACTOR, AGENT, PATIENT, etc., for Penman and FUF), syntactic relations (such as subject, object, premod, etc., for HALogen), or lexi- cal dependencies (Fergus, Amalgam). Such inputs cannot be accurately produced by state-of-the-art analysis components from arbitrary textual input in the context of text-to-text applications. Second, most of the recent systems (starting with Nitrogen) have adopted a hybrid approach to generation, which has increased their robust- ness. These hybrid systems use, in a first phase, symbolic knowledge to (over)generate a large set of candidate realizations, and, in a second phase, statistical knowledge about the target language (such as stochastic language models) to rank the candidate realizations and find the best scoring one. The disadvantage of the hybrid approach – from the perspective of integrating these sys- tems within end-to-end applications – is that the two generation phases cannot be tightly coupled. More precisely, input-driven preferences and tar- get language–driven preferences cannot be inte- grated in a true probabilistic model that can be trained and tuned for maximum performance. In this paper, we propose WIDL-expressions (WIDL stands for Weighted Interleave, Disjunc- tion, and Lock, after the names of the main op- erators) as a representation formalism that facil- itates the integration of a generic sentence real- ization system within end-to-end language appli- cations. The …

Content truncated to 500 words