OCLSP at SemEval-2016 Task 9: Multilayered LSTM as a Neural Semantic Dependency Parser

Proceedings of SemEval-2016, pages 1212–1217, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics OCLSP at SemEval-2016 Task 9: Multilayered LSTM as a Neural Semantic Dependency Parser Lifeng Jin Manjuan Duan William Schuler Department of Linguistics The Ohio State University {jin,duan,schuler}@ling.osu.edu Abstract Semantic dependency parsing aims at ex- tracting arcs and semantic role labels for all words in a sentence. In this paper, we propose a semantic dependency parser which is based on Long Short-term Mem- ory and makes heavy use of embeddings of words and POS tags. We describe in detail the implementation of the neural parser, including preprocessing, postpro- cessing and various input features, and show that the neural parser performs close to the top system in the shared task and is very good at capturing non-local depen- dencies. We also discuss some issues re- lated to the parser and how to improve it. 1 Introduction Semantic dependency parsing is a form of se- mantic analysis where semantic roles for all words in a sentence are analyzed and specific se- mantic relations are assigned to each word pair (Che et al., 2012). Chinese semantic depen- dency parsing is especially of interest as there is a remarkable difference between syntactic and semantic dependencies in Chinese (Che et al., 2012). In the SemEval 2016 Task 9 Chinese Semantic Dependency Parsing shared task, se- mantic dependency parsing for two text genres, TEXT, which includes sentences from conversa- tions and primary school textbooks, and NEWS, which contains newswire text, is explored. We propose a neural parser with LSTM as the basic units and make heavy use of vectorial represen- tations of basic linguistic units like words and POS tags. In this paper, we give a detailed de- scription of the neural parser and discuss a few issues related to the current implementation. 2 Long Short-term Memory Recurrent Neural Networks, or RNNs, are good for sequential prediction, but the problem of exploding or vanishing gradients makes learn- ing long distance dependencies very difficult for them (Hochreiter, 1998). The LSTM ar- chitecture is proposed to address this problem (Hochreiter and Schmidhuber, 1997). In this pa- per, we follow the version defined in Graves et al. (2013b). it = σ(Wxixt + Whiht−1 + Wcict−1 + bi) (1) ft = σ(Wxfxt + Whfht−1 + Wcfct−1 + bf) (2) ct = ftct−1 + it tanh(Wxcxt + Whcht−1 + bc) (3) ot = σ(Wxoxt + Whoht−1 + Wcoct−1 + bo) (4) ht = ot tanh(ct) (5) where σ is the sigmoid function, and i, f, o and c are the input gate, forget gate, output gate and the cell respectively. xt is the input at time step t, and ht is the hidden state or the out- put of the LSTM unit at time step t. With various gates and a cell, the LSTM unit may learn to store and release information inside the cell over many time steps. LSTMs have recently been used for various NLP tasks such as machine translation (Bahdanau et al., 2014; …

Content truncated to 500 words