Towards Robust Context-Sensitive Sentence Alignment for Monolingual Corpora

Towards Robust Context-Sensitive Sentence Alignment for Monolingual Corpora Rani Nelken and Stuart M. Shieber Division of Engineering and Applied Sciences Harvard University 33 Oxford St. Cambridge, MA 02138 � nelken,shieber � @deas.harvard.edu Abstract Aligning sentences belonging to compa- rable monolingual corpora has been sug- gested as a first step towards training text rewriting algorithms, for tasks such as summarization or paraphrasing. We present here a new monolingual sen- tence alignment algorithm, combining a sentence-based TF*IDF score, turned into a probability distribution using logistic re- gression, with a global alignment dynamic programming algorithm. Our approach provides a simpler and more robust solu- tion achieving a substantial improvement in accuracy over existing systems. 1 Introduction Sentence-aligned bilingual corpora are a crucial resource for training statistical machine trans- lation systems. Several authors have sug- gested that large-scale aligned monolingual cor- pora could be similarly used to advance the perfor- mance of monolingual text-to-text rewriting sys- tems, for tasks including summarization (Knight and Marcu, 2000; Jing, 2002) and paraphras- ing (Barzilay and Elhadad, 2003; Quirk et al., 2004). Unlike bilingual corpora, such as the Cana- dian Hansard corpus, which are relatively rare, it is now fairly easy to amass corpora of related mono- lingual documents. For instance, with the ad- vent of news aggregator services such as “Google News”, one can readily collect multiple news sto- ries covering the same news item (Dolan et al., 2004). Utilizing such a resource requires align- ing related documents at a finer level of resolu- tion, identifying which sentences from one docu- ment align with which sentences from the other. Previous work has shown that aligning related monolingual documents is quite different from the well-studied multi-lingual alignment task. Whereas documents in a bilingual corpus are typ- ically very closely aligned, monolingual corpora exhibit a much looser level of alignment, with similar content expressed using widely divergent wording, grammatical form, and sentence order. Consequently, many of the simple surface-based methods that have proven to be so successful in bilingual sentence alignment, such as correlation of sentence length, linearity of alignment, and a predominance of one-to-one sentence mapping, are much less likely to be effective for monolin- gual sentence alignment. Barzilay and Elhadad (2003) suggested that these disadvantages could be at least partially off- set by the recurrence of the same lexical items in document pairs. Indeed, they showed that a sim- ple cosine word-overlap score is a good baseline for the task, outperforming much more sophisti- cated methods. They also observed that context is a powerful factor in determining alignment. They illustrated this on a corpus of Encyclopedia Bri- tannica entries describing world cities, where each entry comes in two flavors, the comprehensive en- cyclopedia entry, and a shorter and simpler ele- mentary version. Barzilay and Elhadad used con- text in two different forms. First, using inter- document context, they took advantage of com- monalities in the topical structure of the encyclo- pedia entries to identify paragraphs that are likely to be about the same topic. They then took …

Content truncated to 500 words