Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1584–1591 June 6–11, 2021. ©2021 Association for Computational Linguistics 1584 Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network Fan Jiang and Trevor Cohn School of Computing and Information Systems The University of Melbourne, Victoria, Australia fan.jiang1@student.unimelb.edu.au t.cohn@unimelb.edu.au Abstract External syntactic and semantic information has been largely ignored by existing neural coreference resolution models. In this pa- per, we present a heterogeneous graph-based model to incorporate syntactic and semantic structures of sentences. The proposed graph contains a syntactic sub-graph where tokens are connected based on a dependency tree, and a semantic sub-graph that contains arguments and predicates as nodes and semantic role la- bels as edges. By applying a graph atten- tion network, we can obtain syntactically and semantically augmented word representation, which can be integrated using an attentive in- tegration layer and gating mechanism. Experi- ments on the OntoNotes 5.0 benchmark show the effectiveness of our proposed model.1 1 Introduction Coreference resolution is a core task in NLP, which aims to identify all mentions that refer to the same entity. Coreference encodes rich semantic infor- mation which has been successfully applied to im- prove many downstream NLP tasks (Luan et al., 2019; Wadden et al., 2019; Dasigi et al., 2019; Sto- janovski and Fraser, 2018). Impressive progress has been made in recent years since the introduction of the first end-to-end neural coreference resolution model (Lee et al., 2017) by utilising contextualized embeddings from large pretrained language models (Joshi et al., 2019, 2020; Kantor and Globerson, 2019; Wu et al., 2020) such as ELMo (Peters et al., 2018) and BERT (De- vlin et al., 2019). Rich language knowledge en- coded in these pretrained models has largely allevi- ated the need for syntactic and semantic features. However, such information has been shown to ben- efit BERT based models on other tasks (Nie et al., 2020a; Wang et al., 2020; Pouran Ben Veyseh et al., 1https://github.com/Fantabulous-J/coref-HGAT 2020). Therefore, we believe such information could also benefit the coreference resolution task. In this paper, we propose a neural coreference resolution model based on Joshi et al. (2019), which we extend by incorporating external syn- tactic and semantic information. For syntactic information, we use dependency trees to capture the long-term dependency exists among mentions. Kong and Jian (2019) has successfully incorpo- rated structural information into neural models, but their model still requires the design of com- plex hand-engineered features. In contrast, our model is more flexible, using a graph neural net- work to encode syntax in the form of dependency trees. For semantic information, we adopt semantic role labelling (SRL) structures. SRL labels capture who did what to whom and it is effective in pro- viding document-level event description informa- tion, which allows us to better identify the relation- ship between event mentions. Previous statistical coreference systems have successfully integrated such information (Ponzetto and Strube, 2006; Kong et al., 2009), but their …

Content truncated to 500 words