Neural End-to-end Coreference Resolution for German in Different Domains

Neural End-to-end Coreference Resolution for German in Different Domains Fynn Schr¨oder∗, Hans Ole Hatzel∗, and Chris Biemann Language Technology Group Universit¨at Hamburg, Germany {fschroeder, hatzel, biemann}@informatik.uni-hamburg.de Abstract We apply neural coreference resolution to German, surpassing the previous state-of-the- art performance by a wide margin of 10– 30 points F1 across three established datasets for German. This is achieved by a neural end-to-end approach, training contextual word- embeddings jointly with mention and entity similarity scores. We explore the impact of var- ious parameters such as language models, pre- training and computational limits with respect to German data. In an effort to support datasets representing the domains of both news and lit- erature, we make use of two distinct model architectures: a mention linking-based and an incremental entity-based approach that should scale to very long documents such as literary works. Our code and ready-to-use models are publicly available. 1 Introduction Coreference resolution is the task of resolving text spans in documents that refer to the same entities. These are grouped into mention-clusters with each cluster representing one entity. Figure 1 shows coreference annotations on a literary text with dif- ferent entities being denoted by both subscripts and colors. Tasks such as question answering (Mor- ton, 1999) or text summarization (Steinberger et al., 2007) can rely on coreference resolution as part of the language processing pipeline. Bamman et al. (2014) demonstrated that coreference resolution is also applicable to literary analysis. The task has recently seen large improvements as systems moved from rule-based (e.g. Roesiger and Kuhn, 2016; Lee et al., 2011) to neural approaches (e.g. Lee et al., 2017; Joshi et al., 2019). This advance- ment from a CoNLL-F1-score of 57.8, achieved by a rule-based system in the original CoNLL-2012 shared task (Pradhan et al., 2012), to 67.2 in the ∗denotes equal contribution [Alice] 1�was�not�a�bit�hurt,�and�[she] 1�jumped�up�on�to [her] 1�feet�in�a�moment:� [she] 1�looked�up,�but�it�was all� dark� overhead;� before� [her] 1� was� [another� long passage] 2,�and� [the�White�Rabbit] 3�was�still�in�sight, hurrying�down�[it] 2. Figure 1: Coreference gold annotations for “Alice’s Ad- ventures in Wonderland” (annotations from Bamman et al., 2020) first end-to-end neural system (Lee et al., 2017) has shown that neural systems are key to state-of-the- art performance. Coreference resolution on German using neural networks has received little attention. There has, to our knowledge, no work been reported on German news datasets using neural networks yet. This work is also the first to use cross-task learning to improve performance on German literary datasets. We apply and adapt exiting approaches to coref- erence on German, making our code and models publicaly available.1 There are two approaches to neural coreference resolution that we consider: A mention-linking-based and an entity-linking-based approach. Both have an initial mention proposal step, finding text spans that are likely to represent mentions. In mention-linking approaches, out of the cross-products of mentions, those mentions with the highest likelihood are considered. Each such mention is connected to its highest-scoring antecedent with transitively connected mentions forming entities. The entity-representation-based approach also involves the initial mention proposal step. How- ever, rather then …

Content truncated to 500 words