Deep Semantic Role Labeling: What Works and What’s Next

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 473–483 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1044 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 473–483 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1044 Deep Semantic Role Labeling: What Works and What’s Next Luheng He†, Kenton Lee†, Mike Lewis‡, and Luke Zettlemoyer†∗ † Paul G. Allen School of Computer Science & Engineering, Univ. of Washington, Seattle, WA {luheng, kentonl, lsz}@cs.washington.edu ‡ Facebook AI Research, Menlo Park, CA mikelewis0@fb.com ∗Allen Institute for Artificial Intelligence, Seattle, WA lukez@allenai.org Abstract We introduce a new deep learning model for semantic role labeling (SRL) that sig- nificantly improves the state of the art, along with detailed analyses to reveal its strengths and limitations. We use a deep highway BiLSTM architecture with con- strained decoding, while observing a num- ber of recent best practices for initializa- tion and regularization. Our 8-layer en- semble model achieves 83.2 F1 on the CoNLL 2005 test set and 83.4 F1 on CoNLL 2012, roughly a 10% relative er- ror reduction over the previous state of the art. Extensive empirical analysis of these gains show that (1) deep models excel at recovering long-distance dependencies but can still make surprisingly obvious errors, and (2) that there is still room for syntactic parsers to improve these results. 1 Introduction Semantic role labeling (SRL) systems aim to re- cover the predicate-argument structure of a sen- tence, to determine essentially “who did what to whom”, “when”, and “where.” Recently break- throughs involving end-to-end deep models for SRL without syntactic input (Zhou and Xu, 2015; Marcheggiani et al., 2017) seem to overturn the long-held belief that syntactic parsing is a pre- requisite for this task (Punyakanok et al., 2008). In this paper, we show that this result can be pushed further using deep highway bidirectional LSTMs with constrained decoding, again signifi- cantly moving the state of the art (another 2 points on CoNLL 2005). We also present a careful em- pirical analysis to determine what works well and what might be done to progress even further. Our model combines a number of best prac- tices in the recent deep learning literature. Fol- lowing Zhou and Xu (2015), we treat SRL as a BIO tagging problem and use deep bidirectional LSTMs. However, we differ by (1) simplifying the input and output layers, (2) introducing high- way connections (Srivastava et al., 2015; Zhang et al., 2016), (3) using recurrent dropout (Gal and Ghahramani, 2016), (4) decoding with BIO- constraints, and (5) ensembling with a product of experts. Our model gives a 10% relative error re- duction over previous state of the art on the test sets of CoNLL 2005 and 2012. We also report per- formance with predicted predicates to encourage future exploration of end-to-end SRL systems. We present detailed error analyses to better un- derstand the performance gains, including (1) de- sign choices on architecture, …

Content truncated to 500 words