Cohere: A Toolkit for Local Coherence

Cohere: A Toolkit for Local Coherence Karin Sim Smith§, Wilker Aziz†, Lucia Specia§ §Department of Computer Science, University of Sheffield, UK †Institute for Logic, Language and Computation University of Amsterdam, The Netherlands {kmsimsmith1,l.specia}@sheffield.ac.uk , w.aziz@uva.nl Abstract We describe cohere, our coherence toolkit which incorporates various complementary models for capturing and measuring different aspects of text coherence. In addition to the traditional entity grid model (Lapata, 2005) and graph-based metric (Guinaudeau and Strube, 2013), we provide an implementation of a state-of-the-art syntax-based model (Louis and Nenkova, 2012), as well as an adaptation of this model which shows significant performance improvements in our experiments. We benchmark these models using the standard setting for text coherence: original documents and versions of the document with sentences in shuffled order. Keywords: Discourse, Coherence Models, Local Coherence 1. Introduction Coherence can be defined as what holds a text together and makes it logical and understandable for the reader. This is achieved by various means, some of which are more amenable to computational models than others. The only freely available coherence software we are aware of is the Brown Coherence Toolkit.1 It provides a range of entity-grid-based coherence models, capturing information related to entities that are common to adjacent sentences. Our toolkit offers additional, more varied models which in- clude syntax, and in the case of the graph-based metric, capture links throughout the text. In what follows, we briefly describe the models in our toolkit (Section 2.), provide information on how to use the toolkit (Section 3.), and show our experiments with it (Sec- tion 4.). 2. Models in cohere We briefly describe our own implementation of the most popular model for text coherence, the entity grid model, in Section 2.1.. Then in Section 2.2. we describe our entity graph implementation, which captures different aspects of lexical coherence from an entity grid, tracking connections between non-adjacent entities in the text. This model cov- ers aspects of coherence that reflect the centrality and topic of the discourse. Following that we present our implemen- tation of the syntax-based coherence model of Louis and Nenkova (2012), which captures syntactic patterns between adjacent sentences (Section 2.3.). Finally, in Section 2.4. we present our own extension of this model, a fully gen- erative model incorporating IBM Model 1 (Brown et al., 1993) to model alignments over syntactic items in adjacent sentences. 2.1. Entity Grid Model As detailed in (Barzilay and Lapata, 2008; Elsner et al., 2007), the entity-based approach derives from the assump- tion that entities in a coherent text are distributed in a cer- tain manner, as posed by various discourse theories, in par- 1http://cs.brown.edu/melsner/manual.html ticular Centering Theory (Grosz et al., 1995). This theory holds that coherent texts are characterised by salient entities in strong grammatical roles, such as subject or object. En- tity grids are constructed by identifying the discourse enti- ties in the documents under consideration, and constructing a 2D grid for each document, whereby each column corre- sponds to the entity, i.e. noun, being tracked, and each row …

Content truncated to 500 words