A Phrase-Based HMM Approach to Document/Abstract Alignment

A Phrase-Based HMM Approach to Document/Abstract Alignment Hal Daum´e III and Daniel Marcu Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 {hdaume,marcu}@isi.edu Abstract We describe a model for creating word-to-word and phrase-to-phrase alignments between documents and their human written abstracts. Such alignments are critical for the development of statistical sum- marization systems that can be trained on large cor- pora of document/abstract pairs. Our model, which is based on a novel Phrase-Based HMM, outper- forms both the Cut & Paste alignment model (Jing, 2002) and models developed in the context of ma- chine translation (Brown et al., 1993). 1 Introduction There are a wealth of document/abstract pairs that statistical summarization systems could leverage to learn how to create novel abstracts. Detailed stud- ies of such pairs (Jing, 2002) show that human ab- stractors perform a range of very sophisticated op- erations when summarizing texts, which include re- ordering, fusion, and paraphrasing. Unfortunately, existing document/abstract alignment models are not powerful enough to capture these operations. To get around directly tackling this problem, re- searchers in text summarization have employed one of several techniques. Some researchers (Banko et al., 2000) have de- veloped simple statistical models for aligning doc- uments and headlines. These models, which imple- ment IBM Model 1 (Brown et al., 1993), treat docu- ments and headlines as simple bags of words and learn probabilistic word-based mappings between the words in the documents and the words in the headlines. As our results show, these models are too weak for capturing the operations that are em- ployed by humans in summarizing texts beyond the headline level. Other researchers have developed models that make unreasonable assumptions about the data, which lead to the utilization of a very small per- cent of available data. For instance, the docu- ment and sentence compression models of Daum´e III, Knight, and Marcu (Knight and Marcu, 2002; Daum´e III and Marcu, 2002a) assume that sen- tences/documents can be summarized only through deletion of contiguous text segments. Knight and Marcu found that from a corpus of 39, 060 abstract sentences, only 1067 sentence extracts existed: a re- call of only 2.7%. An alternate techinque employed in a large vari- ety of systems is to treat the summarization prob- lem as a sentence extraction problem. Such sys- tems can be trained either on human constructed ex- tracts or extracts generated automatically from doc- ument/abstract pairs (see (Marcu, 1999; Jing and McKeown, 1999) for two such approaches). None of these techniques is adequate. Even for a relatively simple sentence from an abstract, we can see that none of the assumptions listed above holds. In Figure 1, we observe several phenomena: • Alignments can occur at the granularity of words and at the granularity of phrases. • The ordering of phrases in an abstract can be different from the ordering in the document. • Some abstract words do not have direct cor- respondents in the document, and some doc- ument words are …

Content truncated to 500 words