An Empirical Study on Finding Spans

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3976 - 3983 December 7-11, 2022 ©2022 Association for Computational Linguistics An Empirical Study on Finding Spans Weiwei Gu1* Boyuan Zheng2* Yunmo Chen2 Tongfei Chen3 Benjamin Van Durme2 1 University of Rochester 2 Johns Hopkins University 3 Microsoft Semantic Machines wgu7@ur.rochester.edu, {bzheng12,yunmo,vandurme}@jhu.edu, tongfei@pm.me Abstract We present an empirical study on methods for span finding, the selection of consecutive to- kens in text for some downstream tasks. We focus on approaches that can be employed in training end-to-end information extraction sys- tems, and find there is no definitive solution without considering task properties, and pro- vide our observations to help with future de- sign choices: 1) a tagging approach often yields higher precision while span enumeration and boundary prediction provide higher recall; 2) span type information can benefit a boundary prediction approach; 3) additional contextual- ization does not help span finding in most cases. 1 Introduction Various information extraction (IE) tasks require a span finding component, which either directly yields the output or serves as an essential com- ponent of downstream linking. In named entity recognition (NER), spans in text are detected and typed; coreference resolution (RE) requires men- tion spans; mention spans are linked when perform- ing relation extraction (RE), and in event extraction this also requires detection of trigger spans. In extractive question answering (QA), a span in a passage is detected to be presented as the answer to a given question. Following the proliferation of large pre-trained models (Peters et al., 2018; Devlin et al., 2019; Raffel et al., 2020, i.a.), recent approaches to span finding can be roughly divided into three different types: as tagging, span enumeration, or bound- ary prediction (see §2, Table 1). In this paper, we present an empirical study of these methods and their influence on downstream tasks, hoping to shed light on how to build future NLP systems. Specifi- cally, we discuss the design choice of span finding methods and examine two common tricks for im- * Equal contribution. North Korea has nuclear weapons B I O B I North Korea has nuclear weapons L R ✓ ✓ ✓ ✓ North Korea has nuclear weapons (a) Tagging (b) Span enumeration (c) Boundary prediction Figure 1: Three common methods for span find- ing, detecting the mention spans North Korea and nuclear weapons in the sentence. “�” denotes a candi- date span that is too long to be considered. proving performance. We answer the following questions: 1. Q: What is the best span finding method? Does this hold for various NLP tasks or different pre- trained encoders? A: The choice depends on the downstream task: tagging generally has higher precision, but span enumeration or boundary prediction has higher recall. For most cases, boundary prediction is preferable to span enumeration. Tagging per- forms much better on a masked language model pretrained encoder (e.g., RoBERTa) than an encoder-decoder pretrained model (e.g., T5). 2. Q: Does inclusion of mention type information help (e.g., just …

Content truncated to 500 words