End-to-end Deep Reinforcement Learning Based Coreference Resolution

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 660–665 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 660 End-to-end Deep Reinforcement Learning Based Coreference Resolution Hongliang Fei, Xu Li, Dingcheng Li, Ping Li Cognitive Computing Lab, Baidu Research {hongliangfei,lixu13,lidingcheng,liping11}@baidu.com Abstract Recent neural network models have signifi- cantly advanced the task of coreference res- olution. However, current neural coreference models are typically trained with heuristic loss functions that are computed over a sequence of local decisions. In this paper, we introduce an end-to-end reinforcement learning based coreference resolution model to directly opti- mize coreference evaluation metrics. Specif- ically, we modify the state-of-the-art higher- order mention ranking approach in Lee et al. (2018) to a reinforced policy gradient model by incorporating the reward associated with a sequence of coreference linking actions. Fur- thermore, we introduce maximum entropy reg- ularization for adequate exploration to pre- vent the model from prematurely converging to a bad local optimum. Our proposed model achieves new state-of-the-art performance on the English OntoNotes v5.0 benchmark. 1 Introduction Coreference resolution is one of the most fun- damental tasks in natural language processing (NLP), which has a significant impact on many downstream applications including information extraction (Dai et al., 2019), question answer- ing (Weston et al., 2015), and entity linking (Ha- jishirzi et al., 2013). Given an input text, corefer- ence resolution aims to identify and group all the mentions that refer to the same entity. In recent years, deep neural network models for coreference resolution have been prevalent (Wise- man et al., 2016; Clark and Manning, 2016b). These models, however, either assumed mentions were given and only developed a coreference link- ing model (Clark and Manning, 2016b) or built a pipeline system to detect mention first then re- solved coreferences (Haghighi and Klein, 2010). In either case, they depend on hand-crafted fea- tures and syntactic parsers that may not generalize well or may even propagate errors. To avoid the cascading errors of pipeline sys- tems, recent NLP researchers have developed end- to-end approaches (Lee et al., 2017; Luan et al., 2018; Lee et al., 2018; Zhang et al., 2018), which directly consider all text spans, jointly identify entity mentions and cluster them. The core of those end-to-end models are vector embeddings to represent text spans in the document and scor- ing functions to compute the mention scores for text spans and antecedent scores for pairs of spans. Depending on how the span embeddings are com- puted, the end-to-end coreference models could be further divided into first order methods (Lee et al., 2017; Luan et al., 2018; Zhang et al., 2018) or higher order methods (Lee et al., 2018). Although recent end-to-end neural coreference models have advanced the state-of-the-art perfor- mance for coreference resolution, they are still trained with heuristic loss functions and make a sequence of local decisions for each pair of men- tions. However as studied in Clark and Manning (2016a); Yin et al. (2018), most coreference reso- lution …

Content truncated to 500 words