Adapting Coreference Resolution Models through Active Learning

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7533 - 7549 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Adapting Coreference Resolution Models through Active Learning Michelle Yuan† Patrick Xia‡ Chandler May‡ Benjamin Van Durme‡ Jordan Boyd-Graber† Human Language Technology Center of Excellence †University of Maryland ‡Johns Hopkins University myuan@cs.umd.edu paxia@cs.jhu.edu jbg@umiacs.umd.edu Abstract Neural coreference resolution models trained on one dataset may not transfer to new, low- resource domains. Active learning mitigates this problem by sampling a small subset of data for annotators to label. While active learn- ing is well-defined for classification tasks, its application to coreference resolution is neither well-defined nor fully understood. This pa- per explores how to actively label coreference, examining sources of model uncertainty and document reading costs. We compare uncer- tainty sampling strategies and their advantages through thorough error analysis. In both syn- thetic and human experiments, labeling spans within the same document is more effective than annotating spans across documents. The findings contribute to a more realistic develop- ment of coreference resolution models. 1 Introduction Linguistic expressions are coreferent if they refer to the same entity. The computational task of dis- covering coreferent mentions is coreference resolu- tion (CR). Neural models (Lee et al., 2018; Joshi et al., 2020) are SOTA on ONTONOTES 5.0 (Pradhan et al., 2013) but cannot immediately generalize to other datasets. Generalization is difficult be- cause domains differ in content, writing style, and annotation guidelines. To overcome these chal- lenges, models need copiously labeled, in-domain data (Bamman et al., 2020). Despite expensive labeling costs, adapting CR is crucial for applications like uncovering infor- mation about proteins in biomedicine (Kim et al., 2012) and distinguishing entities in legal docu- ments (Gupta et al., 2018). Ideally, we would like to quickly and cheaply adapt the model without repeatedly relying on an excessive amount of an- notations to retrain the model. To reduce labeling cost, we investigate active learning (Settles, 2009) for CR. Active learning aims to reduce annotation costs by intelligently selecting examples to label. Prior approaches use active learning to improve the model within the same domain (Gasperin, 2009; Sachan et al., 2015) without considering adapting to new data distributions. For domain adaptation in CR, Zhao and Ng (2014) motivate the use of active learning to select out-of-distribution exam- ples. A word like “the bonds” refers to municipal bonds in ONTONOTES but links to “chemical bonds” in another domain (Figure 1). If users annotate the antecedents of “the bonds” and other ambigu- ous entity mentions, then these labels help adapt a model trained on ONTONOTES to new domains. Active learning for CR adaptation is well- motivated, but the implementation is neither straightforward nor well-studied. First, CR is a span detection and clustering task, so selecting which spans to label is more complicated than choos- ing independent examples for text classification. Second, CR labeling involves closely reading the documents. Labeling more spans within the same context is more efficient. However, labeling more …

Content truncated to 500 words