A Structured Span Selector

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2629 - 2641 July 10-15, 2022 ©2022 Association for Computational Linguistics A Structured Span Selector Tianyu Liu Yuchen Eleanor Jiang Ryan Cotterell Mrinmaya Sachan {tianyu.liu, yucjiang, ryan.cotterell, mrinmaya.sachan}@inf.ethz.ch Abstract Many natural language processing tasks, e.g., coreference resolution and semantic role label- ing, require selecting text spans and making decisions about them. A typical approach to such tasks is to score all possible spans and greedily select spans for task-specific downstream processing. This approach, how- ever, does not incorporate any inductive bias about what sort of spans ought to be selected, e.g., that selected spans tend to be syntactic constituents. In this paper, we propose a novel grammar-based structured span selection model which learns to make use of the partial span-level annotation provided for such prob- lems. Compared to previous approaches, our approach gets rid of the heuristic greedy span selection scheme, allowing us to model the downstream task on an optimal set of spans. We evaluate our model on two popular span prediction tasks: coreference resolution and semantic role labeling. We show empirical improvements on both. https://github.com/lyutyuh/ structured-span-selector 1 Introduction The problem of selecting a continuous segment of the input text, termed a span, is a common de- sign pattern1 in NLP. In this work, we call this design pattern span selection. Common tasks that have a span prediction component include corefer- ence resolution (Stede, 2011), where the selected spans are mentions, semantic role labeling (Palmer et al., 2010), where the selected spans are argu- ments, question answering (Jurafsky and Martin, 2009), where the selected spans are answers, and named entity recognition (Smith, 2011), where the selected spans are entities. In most of the tasks mentioned above, span selec- tion is the first step.2 After a set of candidate spans 1A software engineering metaphor, which refers to a reusable solution to a commonly occurring problem. 2Note that we also have joint models where we find optimal Coref: [The President]1 has said �[he]1 and [his]1 wife, now a [New York]3 senator � 2 will spend weekends at [their]2 house in Chappaqua . SRL: [The most important thing about Disney]1-ARG1 is1 [that [it]2-ARG1 is2 [a global brand]2-ARG2 ]1-ARG2 . Figure 1: Examples of two span prediction tasks: Coreference and SRL. In Coref, [s]i denotes a span of text s referring to the ith entity. In SRL, i denotes a set of predicates and [s]i−∗ denotes a set of arguments for the ith predicate. is determined, a classifier (often a neural network) is typically used to make predictions about the can- didate spans. For instance, in coreference resolu- tion, the selected spans (mentions) are clustered according to which entity they refer to; whereas in SRL, the spans (arguments) are classified into a set of roles. Two examples are shown in Fig. 1. As the number of spans to consider in the in- put text can be quadratic in the length of the input, …

Content truncated to 500 words