Semi-Supervised Semantic Role Labeling with Cross-View Training

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1018–1027, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1018 Semi-Supervised Semantic Role Labeling with Cross-View Training Rui Cai and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB Rui.Cai@ed.ac.uk mlap@inf.ed.ac.uk Abstract The successful application of neural networks to a variety of NLP tasks has provided strong impetus to develop end-to-end models for se- mantic role labeling which forego the need for extensive feature engineering. Recent approaches rely on high-quality annotations which are costly to obtain, and mostly unavail- able in low resource scenarios (e.g., rare lan- guages or domains). Our work aims to re- duce the annotation effort involved via semi- supervised learning. We propose an end-to- end SRL model and demonstrate it can effec- tively leverage unlabeled data under the cross- view training modeling paradigm. Our LSTM- based semantic role labeler is jointly trained with a sentence learner, which performs POS tagging, dependency parsing, and predicate identification which we argue are critical to learning directly from unlabeled data without recourse to external pre-processing tools. Ex- perimental results on the CoNLL-2009 bench- mark dataset show that our model outperforms the state of the art in English, and consistently improves performance in other languages, in- cluding Chinese, German, and Spanish. 1 Introduction Semantic role labeling — the task of automatically identifying and labeling the semantic roles con- veyed by sentential constituents — has enjoyed renewed interest in the last few years thanks to the popularity of neural network models and their ability to learn continuous representations which forego the need for extensive feature engineer- ing. Recent modeling developments aside, seman- tic role labeling (SRL) has been generally recog- nized as a core task in NLP with relevance for ap- plications ranging from machine translation (Aziz et al., 2011; Marcheggiani et al., 2018), to infor- mation extraction (Christensen et al., 2011), and summarization (Khan et al., 2015). State-of-the art semantic role labelers (He et al., 2018b; Cai et al., 2018) rely on high-quality an- notations (of semantic predicates and their argu- ments) for use in training. These annotations are costly to obtain, and mostly unavailable in low re- source scenarios (e.g., rare languages or domains) motivating the need for effective semi-supervised methods that leverage unlabeled examples. Cross- View Training (CVT; Clark et al. 2018) is a re- cently proposed semi-supervised learning algo- rithm that improves representation learning using a mix of labeled and unlabeled data. The main idea behind CVT is to train a model to produce consis- tent predictions across different restricted views of the input with the aid of auxiliary prediction tasks. Clark et al. (2018) demonstrate performance gains when applying CVT to sequence tagging tasks, machine translation, and dependency parsing. Unfortunately, application of CVT to seman- tic role labeling is fraught with difficulty. This is partly due to the nature of …

Content truncated to 500 words