A Neural Network for Coordination Boundary Prediction

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 23–32, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics A Neural Network for Coordination Boundary Prediction Jessica Ficler Computer Science Department Bar-Ilan University Israel jessica.ficler@gmail.com Yoav Goldberg Computer Science Department Bar-Ilan University Israel yoav.goldberg@gmail.com Abstract We propose a neural-network based model for coordination boundary prediction. The net- work is designed to incorporate two signals: the similarity between conjuncts and the ob- servation that replacing the whole coordina- tion phrase with a conjunct tends to produce a coherent sentences. The modeling makes use of several LSTM networks. The model is trained solely on conjunction annotations in a Treebank, without using external resources. We show improvements on predicting coor- dination boundaries on the PTB compared to two state-of-the-art parsers; as well as im- provement over previous coordination bound- ary prediction systems on the Genia corpus. 1 Introduction Coordination is a common syntactic phenomena, ap- pearing in 38.8% of the sentences in the Penn Tree- bank (PTB) (Marcus et al., 1993), and in 60.71% of the sentences in the Genia Treebank (Ohta et al., 2002). However, predicting the correct conjuncts span remain one of the biggest challenges for state- of-the-art syntactic parsers. Both the Berkeley and Zpar phrase-structure parsers (Petrov et al., 2006; Zhang and Clark, 2011) achieve F1 scores of around 69% when evaluated on their ability to recover coor- dination boundaries on the PTB test set. For exam- ple, in: “He has the government’s blessing to [build churches] and [spread Unificationism] in that country.” the conjuncts are incorrectly predicted by both parsers: Berkeley: “He has the government’s blessing to [build churches] and [spread Unificationism in that country].” Zpar: “He [has the government’s blessing to build churches] and [spread Unificationism in that country].” In this work we focus on coordination boundary prediction, and suggest a specialized model for this task. We treat it as a ranking task, and learn a scor- ing function over conjuncts candidates such that the correct candidate pair is scored above all other can- didates. The scoring model is a neural network with two LSTM-based components, each modeling a dif- ferent linguistic principle: (1) conjuncts tend to be similar (“symmetry”); and (2) replacing the coor- dination phrase with each of the conjuncts usually result in a coherent sentence (“replacement”). The symmetry component takes into account the con- juncts’ syntactic structures, allowing to capture sim- ilarities that occur in different levels of the syntac- tic structure. The replacement component considers the coherence of the sequence that is produced when connecting the participant parts. Both of these sig- nals are syntactic in nature, and are learned solely based on information in the Penn Treebank. Our model substantially outperforms both the Berkeley and Zpar parsers on the coordination prediction task, while using the exact same training corpus. Seman- tic signals (which are likely to be based on resources external to the treebank) are also relevant for coor- dination disambiguation (Kawahara and Kurohashi, 2008; Hogan, 2007) and …

Content truncated to 500 words