End-to-end Neural Coreference Resolution

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188–197 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics End-to-end Neural Coreference Resolution Kenton Lee†, Luheng He†, Mike Lewis‡, and Luke Zettlemoyer†∗ † Paul G. Allen School of Computer Science & Engineering, Univ. of Washington, Seattle, WA ∗ Allen Institute for Artificial Intelligence, Seattle WA {kentonl, luheng, lsz}@cs.washington.edu ‡ Facebook AI Research, Menlo Park, CA mikelewis@fb.com Abstract We introduce the first end-to-end corefer- ence resolution model and show that it sig- nificantly outperforms all previous work without using a syntactic parser or hand- engineered mention detector. The key idea is to directly consider all spans in a document as potential mentions and learn distributions over possible antecedents for each. The model computes span em- beddings that combine context-dependent boundary representations with a head- finding attention mechanism. It is trained to maximize the marginal likelihood of gold antecedent spans from coreference clusters and is factored to enable aggres- sive pruning of potential mentions. Exper- iments demonstrate state-of-the-art perfor- mance, with a gain of 1.5 F1 on the OntoNotes benchmark and by 3.1 F1 us- ing a 5-model ensemble, despite the fact that this is the first approach to be success- fully trained with no external resources. 1 Introduction We present the first state-of-the-art coreference resolution model that is learned end-to-end given only gold mention clusters. All recent coref- erence models, including neural approaches that achieved impressive performance gains (Wiseman et al., 2016; Clark and Manning, 2016b,a), rely on syntactic parsers, both for head-word features and as the input to carefully hand-engineered men- tion proposal algorithms. We demonstrate for the first time that these resources are not required, and in fact performance can be improved significantly without them, by training an end-to-end neural model that jointly learns which spans are entity mentions and how to best cluster them. Our model reasons over the space of all spans up to a maximum length and directly optimizes the marginal likelihood of antecedent spans from gold coreference clusters. It includes a span-ranking model that decides, for each span, which of the previous spans (if any) is a good antecedent. At the core of our model are vector embeddings rep- resenting spans of text in the document, which combine context-dependent boundary representa- tions with a head-finding attention mechanism over the span. The attention component is in- spired by parser-derived head-word matching fea- tures from previous systems (Durrett and Klein, 2013), but is less susceptible to cascading errors. In our analyses, we show empirically that these learned attention weights correlate strongly with traditional headedness definitions. Scoring all span pairs in our end-to-end model is impractical, since the complexity would be quartic in the document length. Therefore we fac- tor the model over unary mention scores and pair- wise antecedent scores, both of which are sim- ple functions of the learned span embedding. The unary mention scores are used to prune the space of spans and antecedents, to aggressively reduce the number of …

Content truncated to 500 words