Exploiting Mutual Benefits between Syntax and Semantic Roles using Neural Network

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 968–974, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Exploiting Mutual Benefits between Syntax and Semantic Roles using Neural Network Peng Shi‡∗ Zhiyang Teng† Yue Zhang† †Singapore University of Technology and Design (SUTD) ‡Zhejiang University, China impavidity@zju.edu.cn zhiyang teng@mymail.sutd.edu.sg, yue zhang@sutd.edu.sg Abstract We investigate mutual benefits between syn- tax and semantic roles using neural network models, by studying a parsing→SRL pipeline, a SRL→parsing pipeline, and a simple joint model by embedding sharing. The integra- tion of syntactic and semantic features gives promising results in a Chinese Semantic Tree- bank, demonstrating large potentials of neural models for joint parsing and semantic role la- beling. 1 Introduction The correlation between syntax and semantics has been a fundamental problem in natural language processing (Steedman, 2000). As a shallow seman- tic task, semantic role labeling (SRL) models have traditionally been built upon syntactic parsing re- sults (Gildea and Jurafsky, 2002; Gildea and Palmer, 2002; Punyakanok et al., 2005). It has been shown that parser output features play a crucial role for ac- curate SRL (Pradhan et al., 2005; Surdeanu et al., 2007). On the reverse direction, semantic role features have been used to improve parsing (Boxwell et al., 2010). Existing methods typically use semantic fea- tures to rerank n-best lists of syntactic parsing mod- els (Surdeanu et al., 2008; Hajiˇc et al., 2009). There has also been attempts to learn syntactic parsing and semantic role labeling models jointly, but most such efforts have led to negative results (Sutton and Mc- Callum, 2005; Van Den Bosch et al., 2012; Boxwell et al., 2010). ∗Work done while the first author was visiting SUTD. With the rise of deep learning, neural network models have been used for semantic role label- ing (Collobert et al., 2011). Recently, it has been shown that a neural semantic role labeler can give state-of-the-art accuracies without using parser out- put features, thanks to the use of recurrent neural network structures that automatically capture syn- tactic information (Zhou and Xu, 2015; Wang et al., 2015). In the parsing domain, neural network mod- els have also been shown to give state-of-the-art re- sults recently (Dyer et al., 2015; Weiss et al., 2015; Zhou et al., 2015). The availability of parser-independent neural SRL models allows parsing and SRL to be performed by both parsing→SRL and SRL→parsing pipelines, and gives rise to the interesting research question whether mutual benefits between syntax and seman- tic roles can be better exploited under the neural setting. Different from traditional models that rely on manual feature combinations for joint learning tasks (Sutton and McCallum, 2005; Zhang and Clark, 2008a; Finkel and Manning, 2009; Lewis et al., 2015), neural network models induce non-linear feature combinations automatically from input word and Part-of-Speech (POS) embeddings. This al- lows more complex feature sharing between multi- ple tasks to be achieved effectively (Collobert et al., 2011). We take a first step1 in such investigation by cou- 1Recently, Swayamdipta et al. …

Content truncated to 500 words