A Neural Local Coherence Model

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1320–1330 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1121 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1320–1330 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1121 A Neural Local Coherence Model Dat Tien Nguyen∗ Informatics Institute University of Amsterdam t.d.nguyen@uva.nl Shafiq Joty Qatar Computing Research Institute HBKU, Qatar Foundation sjoty@hbku.edu.qa Abstract We propose a local coherence model based on a convolutional neural network that op- erates over the entity grid representation of a text. The model captures long range en- tity transitions along with entity-specific features without loosing generalization, thanks to the power of distributed repre- sentation. We present a pairwise ranking method to train the model in an end-to-end fashion on a task and learn task-specific high level features. Our evaluation on three different coherence assessment tasks demonstrates that our model achieves state of the art results outperforming existing models by a good margin. 1 Introduction and Motivation What distinguishes a coherent text from a random sequence of sentences is that it binds the sentences together to express a meaning as a whole — the in- terpretation of a sentence usually depends on the meaning of its neighbors. Coherence models that can distinguish a coherent from incoherent texts have a wide range of applications in text genera- tion, summarization, and coherence scoring. Several formal theories of coherence have been proposed (Mann and Thompson, 1988a; Grosz et al., 1995; Asher and Lascarides, 2003), and their principles have inspired development of many existing coherence models (Barzilay and Lap- ata, 2008; Lin et al., 2011; Li and Hovy, 2014). Among these models, the entity grid (Barzilay and Lapata, 2008), which is based on Centering The- ory (Grosz et al., 1995), is arguably the most pop- ular, and has seen a number of improvements over the years. As shown in Figure 1, the entity grid model represents a text by a grid that captures how ∗Both authors contributed equally to this work. grammatical roles of different entities change from sentence to sentence. The grid is then converted into a feature vector containing probabilities of local entity transitions, which enables machine learning models to learn the degree of text coher- ence. Extensions of this basic grid model incorpo- rate entity-specific features (Elsner and Charniak, 2011), multiple ranks (Feng and Hirst, 2012), and coherence relations (Feng et al., 2014). While the entity grid and its extensions have been successful in many applications, they are limited in several ways. First, they use discrete representation for grammatical roles and features, which prevents the model from considering suffi- ciently long transitions (Bengio et al., 2003). Sec- ond, feature vector computation in existing models is decoupled from the target task, which limits the model’s capacity to learn task-specific features. In this paper, we propose a neural architecture for coherence assessment that can capture long range entity transitions along with …

Content truncated to 500 words