Learning Structured Perceptrons for Coreference Resolution with Latent Antecedents and Non-local Features

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 47–57, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Learning Structured Perceptrons for Coreference Resolution with Latent Antecedents and Non-local Features Anders Bj¨orkelund and Jonas Kuhn Institute for Natural Language Processing University of Stuttgart {anders,jonas}@ims.uni-stuttgart.de Abstract We investigate different ways of learning structured perceptron models for coref- erence resolution when using non-local features and beam search. Our experi- mental results indicate that standard tech- niques such as early updates or Learning as Search Optimization (LaSO) perform worse than a greedy baseline that only uses local features. By modifying LaSO to de- lay updates until the end of each instance we obtain significant improvements over the baseline. Our model obtains the best results to date on recent shared task data for Arabic, Chinese, and English. 1 Introduction This paper studies and extends previous work us- ing the structured perceptron (Collins, 2002) for complex NLP tasks. We show that for the task of coreference resolution the straightforward combi- nation of beam search and early update (Collins and Roark, 2004) falls short of more limited fea- ture sets that allow for exact search. This contrasts with previous work on, e.g., syntactic parsing (Collins and Roark, 2004; Huang, 2008; Zhang and Clark, 2008) and linearization (Bohnet et al., 2011), and even simpler structured prediction problems, where early updates are not even nec- essary, such as part-of-speech tagging (Collins, 2002) and named entity recognition (Ratinov and Roth, 2009). The main reason why early updates underper- form in our setting is that the task is too difficult and that the learning algorithm is not able to profit from all training data. Put another way, early up- dates happen too early, and the learning algorithm rarely reaches the end of the instances as it halts, updates, and moves on to the next instance. An alternative would be to continue decod- ing the same instance after the early updates, which is equivalent to Learning as Search Opti- mization (LaSO; Daum´e III and Marcu (2005b)). The learning task we are tackling is however further complicated since the target structure is under-determined by the gold standard annotation. Coreferent mentions in a document are usually an- notated as sets of mentions, where all mentions in a set are coreferent. We adopt the recently pop- ularized approach of inducing a latent structure within these sets (Fernandes et al., 2012; Chang et al., 2013; Durrett and Klein, 2013). This approach provides a powerful boost to the performance of coreference resolvers, but we find that it does not combine well with the LaSO learning strategy. We therefore propose a modification to LaSO, which delays updates until after each instance. The com- bination of this modification with non-local fea- tures leads to further improvements in the cluster- ing accuracy, as we show in evaluation results on all languages from the CoNLL 2012 Shared Task – Arabic, Chinese, and English. We obtain the best results to date on these data …

Content truncated to 500 words