Shift-Reduce Constituent Parsing with Neural Lookahead Features

Shift-Reduce Constituent Parsing with Neural Lookahead Features Jiangming Liu and Yue Zhang Singapore University of Technology and Design, 8 Somapah Road, Singapore, 487372 {jiangming liu, yue zhang}@sutd.edu.sg Abstract Transition-based models can be fast and accu- rate for constituent parsing. Compared with chart-based models, they leverage richer fea- tures by extracting history information from a parser stack, which consists of a sequence of non-local constituents. On the other hand, during incremental parsing, constituent infor- mation on the right hand side of the current word is not utilized, which is a relative weak- ness of shift-reduce parsing. To address this limitation, we leverage a fast neural model to extract lookahead features. In particular, we build a bidirectional LSTM model, which leverages full sentence information to predict the hierarchy of constituents that each word starts and ends. The results are then passed to a strong transition-based constituent parser as lookahead features. The resulting parser gives 1.3% absolute improvement in WSJ and 2.3% in CTB compared to the baseline, giv- ing the highest reported accuracies for fully- supervised parsing. 1 Introduction Transition-based constituent parsers are fast and ac- curate, performing incremental parsing using a se- quence of state transitions in linear time. Pioneer- ing models rely on a classifier to make local de- cisions, searching greedily for local transitions to build a parse tree (Sagae and Lavie, 2005). Zhu et al. (2013) use a beam search framework, which preserves linear time complexity of greedy search, while alleviating the disadvantage of error propaga- tion. The model gives state-of-the-art accuracies at a speed of 89 sentences per second on the standard WSJ benchmark (Marcus et al., 1993). Zhu et al. (2013) exploit rich features by extract- ing history information from a parser stack, which consists of a sequence of non-local constituents. However, due to the incremental nature of shift- reduce parsing, the right-hand side constituents of the current word cannot be used to guide the action at each step. Such lookahead features (Tsuruoka et al., 2011) correspond to the outside scores in chart parsing (Goodman, 1998), which has been effective for obtaining improved accuracies. To leverage such information for improving shift- reduce parsing, we propose a novel neural model to predict the constituent hierarchy related to each word before parsing. Our idea is inspired by the work of Roark and Hollingshead (2009) and Zhang et al. (2010b), which shows that shallow syntactic information gathered over the word sequence can be utilized for pruning chart parsers, improving chart parsing speed without sacrificing accuracies. For ex- ample, Roark and Hollingshead (2009) predict con- stituent boundary information on words as a pre- processing step, and use such information to prune the chart. Since such information is much lighter- weight compared to full parsing, it can be predicted relatively accurately using sequence labellers. Different from Roark and Hollingshead (2009), we collect lookahead constituent information for shift-reduce parsing, rather than pruning informa- tion for chart parsing. Our main concern is improv- ing the accuracy rather than improving the speed. Accordingly, our …

Content truncated to 500 words