Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1416–1426, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution Sam Wiseman1 Alexander M. Rush1,2 1School of Engineering and Applied Sciences Harvard University Cambridge, MA, USA {swiseman,srush,shieber}@seas.harvard.edu Stuart M. Shieber1 Jason Weston2 2Facebook AI Research New York, NY, USA jase@fb.com Abstract We introduce a simple, non-linear mention-ranking model for coreference resolution that attempts to learn distinct feature representations for anaphoricity detection and antecedent ranking, which we encourage by pre-training on a pair of corresponding subtasks. Although we use only simple, unconjoined features, the model is able to learn useful representa- tions, and we report the best overall score on the CoNLL 2012 English test set to date. 1 Introduction One of the major challenges associated with re- solving coreference is that in typical documents the number of mentions (syntactic units capable of referring or being referred to) that are non- anaphoric – that is, that are not coreferent with any previous mention – far exceeds the number of mentions that are anaphoric (Kummerfeld and Klein, 2013; Durrett and Klein, 2013). This preponderance of non-anaphoric mentions makes coreference resolution challenging, partly because many basic coreference features, such as those looking at head, number, or gender match fail to distinguish between truly coreferent pairs and the large number of matching but nonethe- less non-coreferent pairs. Indeed, several au- thors have noted that it is difficult to obtain good performance on the coreference task using sim- ple features (Lee et al., 2011; Fernandes et al., 2012; Durrett and Klein, 2013; Kummerfeld and Klein, 2013; Bj¨orkelund and Kuhn, 2014) and, as a result, state-of-the-art systems tend to use lin- ear models with complicated feature conjunction schemes in order to capture more fine-grained in- teractions. While this approach has shown suc- cess, it is not obvious which additional feature conjunctions will lead to improved performance, which is problematic as systems attempt to scale with new data and features. In this work, we propose a data-driven model for coreference that does not require pre- specifying any feature relationships. Inspired by recent work in learning representations for nat- ural language tasks (Collobert et al., 2011), we explore neural network models which take only raw, unconjoined features as input, and attempt to learn intermediate representations automatically. In particular, the model we describe attempts to create independent feature representations useful for both detecting the anaphoricity of a mention (that is, whether or not a mention is anaphoric) and ranking the potential antecedents of an anaphoric mention. Adequately capturing anaphoricity in- formation has long been thought to be an impor- tant aspect of the coreference task (see Ng (2004) and Section 7), since a strong non-anaphoric sig- nal might, for instance, discourage the erroneous prediction of an antecedent for a non-anaphoric mention even in the presence of a misleading head match. We furthermore attempt to encourage the …

Content truncated to 500 words