A Unified Local and Global Model for Discourse Coherence

Proceedings of NAACL HLT 2007, pages 436–443, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics A Unified Local and Global Model for Discourse Coherence Micha Elsner, Joseph Austerweil, and Eugene Charniak Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University Providence, RI 02912 {melsner,ec}@cs.brown.edu, joseph.austerweil@gmail.com Abstract We present a model for discourse co- herence which combines the local entity- based approach of (Barzilay and Lapata, 2005) and the HMM-based content model of (Barzilay and Lee, 2004). Unlike the mixture model of (Soricut and Marcu, 2006), we learn local and global features jointly, providing a better theoretical ex- planation of how they are useful. As the local component of our model we adapt (Barzilay and Lapata, 2005) by relaxing independence assumptions so that it is ef- fective when estimated generatively. Our model performs the ordering task compet- itively with (Soricut and Marcu, 2006), and significantly better than either of the models it is based on. 1 Introduction Models of coherent discourse are central to several tasks in natural language processing: such mod- els have been used in text generation (Kibble and Power, 2004) and evaluation of human-produced text in educational applications (Miltsakaki and Ku- kich, 2004; Higgins et al., 2004). Moreover, an ac- curate model can reveal information about document structure, aiding in such tasks as supervised summa- rization (Barzilay and Lapata, 2005). Models of coherence tend to fall into two classes. Local models (Lapata, 2003; Barzilay and Lapata, 2005; Foltz et al., 1998) attempt to capture the gen- eralization that adjacent sentences often have similar content, and therefore tend to contain related words. Models of this type are good at finding sentences that belong near one another in the document. How- ever, they have trouble finding the beginning or end of the document, or recovering from sudden shifts in topic (such as occur at paragraph boundaries). Some local models also have trouble deciding which of a pair of related sentences ought to come first. In contrast, the global HMM model of Barzilay and Lee (2004) tries to track the predictable changes in topic between sentences. This gives it a pro- nounced advantage in ordering sentences, since it can learn to represent beginnings, ends and bound- aries as separate states. However, it has no local features; the particular words in each sentence are generated based only on the current state of the doc- ument. Since information can pass from sentence to sentence only in this restricted manner, the model sometimes fails to place sentences next to the correct neighbors. We attempt here to unify the two approaches by constructing a model with both sentence-to-sentence dependencies providing local cues, and a hidden topic variable for global structure. Our local fea- tures are based on the entity grid model of (Barzilay and Lapata, 2005; Lapata and Barzilay, 2005). This model has previously been most successful in a con- ditional setting; to integrate it into our model, we first relax its independence assumptions to improve its performance when used generatively. Our global model …

Content truncated to 500 words