A Constrained Latent Variable Model for Coreference Resolution

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 601–612, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Constrained Latent Variable Model for Coreference Resolution Kai-Wei Chang Rajhans Samdani Dan Roth University of Illinois at Urbana-Champaign {kchang10|rsamdan2|danr}@illinois.edu Abstract Coreference resolution is a well known clus- tering task in Natural Language Processing. In this paper, we describe the Latent Left Linking model (L3M), a novel, principled, and linguis- tically motivated latent structured prediction approach to coreference resolution. We show that L3M admits efficient inference and can be augmented with knowledge-based constraints; we also present a fast stochastic gradient based learning. Experiments on ACE and Ontonotes data show that L3M and its constrained ver- sion, CL3M, are more accurate than several state-of-the-art approaches as well as some structured prediction models proposed in the literature. 1 Introduction Coreference resolution is a challenging task, that in- volves identification and clustering of noun phrases mentions that refer to the same real-world entity. Most machine learning approaches to coreference resolution learn a scoring function to estimate the compatibility between two mentions or two sets of previously clustered mentions. Then, a decoding al- gorithm is designed to aggregate these scores and find an optimal clustering assignment. The most popular of these frameworks is the pair- wise mention model (Soon et al., 2001; Ng and Cardie, 2002; Bengtson and Roth, 2008), which learns a compatibility score of mention-pairs and uses these pairwise scores to obtain a global cluster- ing. Recently, efforts have been made (Haghighi and Klein, 2010; Rahman and Ng, 2011b; Rahman and Ng, 2011c) to consider models that capture higher order interactions, in particular, between mentions and previously identified entities (that is, between mentions and clusters). While such models are po- tentially more expressive, they are largely based on heuristics to achieve computational tractability. This paper focuses on a novel and principled ma- chine learning framework that pushes the state-of- the-art while operating at a mention-pair granularity. We present two models — the Latent Left-Linking Model (L3M), and a version of that is augmented with domain knowledge-based constraints, the Con- strained Latent Left-Linking Model (CL3M). L3M admits efficient inference, linking each mention to a previously occurring mention to its left, much like the existing best-left-link inference models (Ng and Cardie, 2002; Bengtson and Roth, 2008). How- ever, unlike previous best-link techniques, learning in our case is performed jointly with decoding — we present a novel latent structural SVM approach, op- timized using a fast stochastic gradient-based tech- nique. Furthermore, we present a probabilistic gen- eralization of L3M that is more expressive in that it is capable of considering mention-entity interac- tions using scores at the mention-pair granularity. We augment this model with a temperature-like pa- rameter (Samdani et al., 2012) to provide additional flexibility. CL3M augments L3M with knowledge-based constraints following (Roth and Yih, 2004; Denis and Baldridge, 2007). This capability is very de- sirable as shown by the success of the rule-based de- terministic approach of Raghunathan …

Content truncated to 500 words