Structured Tuning for Semantic Role Labeling

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8402–8412 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8402 Structured Tuning for Semantic Role Labeling Tao Li University of Utah tli@cs.utah.edu Parth Anand Jawale University of Colorado Parth.Jawale@colorado.edu Martha Palmer University of Colorado Martha.Palmer@colorado.edu Vivek Srikumar University of Utah svivek@cs.utah.edu Abstract Recent neural network-driven semantic role la- beling (SRL) systems have shown impressive improvements in F1 scores. These improve- ments are due to expressive input representa- tions, which, at least at the surface, are or- thogonal to knowledge-rich constrained decod- ing mechanisms that helped linear SRL mod- els. Introducing the benefits of structure to inform neural models presents a methodolog- ical challenge. In this paper, we present a structured tuning framework to improve mod- els using softened constraints only at training time. Our framework leverages the expressive- ness of neural networks and provides supervi- sion with structured loss components. We start with a strong baseline (RoBERTa) to validate the impact of our approach, and show that our framework outperforms the baseline by learn- ing to comply with declarative constraints. Ad- ditionally, our experiments with smaller train- ing sizes show that we can achieve consistent improvements under low-resource scenarios. 1 Introduction Semantic Role Labeling (SRL, Palmer et al., 2010) is the task of labeling semantic arguments of pred- icates in sentences to identify who does what to whom. Such representations can come in handy in tasks involving text understanding, such as coref- erence resolution (Ponzetto and Strube, 2006) and reading comprehension (e.g., Berant et al., 2014; Zhang et al., 2020). This paper focuses on the question of how knowledge can influence modern semantic role labeling models. Linguistic knowledge can help SRL models in several ways. For example, syntax can drive feature design (e.g., Punyakanok et al., 2005; Toutanova et al., 2005; Kshirsagar et al., 2015; Johansson and Nugues, 2008, and others), and can also be em- bedded into neural network architectures (Strubell et al., 2018). In addition to such influences on input represen- tations, knowledge about the nature of semantic roles can inform structured decoding algorithms used to construct the outputs. The SRL literature is witness to a rich array of techniques for structured inference, including integer linear programs (e.g., Punyakanok et al., 2005, 2008), bespoke inference algorithms (e.g., T¨ackstr¨om et al., 2015), A* decod- ing (e.g., He et al., 2017), greedy heuristics (e.g., Ouchi et al., 2018), or simple Viterbi decoding to ensure that token tags are BIO-consistent. By virtue of being constrained by the definition of the task, global inference promises semantically meaningful outputs, and could provide valuable signal when models are being trained. However, beyond Viterbi decoding, it may impose prohibitive computational costs, thus ruling out using infer- ence during training. Indeed, optimal inference may be intractable, and inference-driven training may require ignoring certain constraints that render inference difficult. While global inference was a mainstay of SRL models until recently, today’s end-to-end trained neural architectures have shown remarkable suc- cesses without needing …

Content truncated to 500 words