Structural Embedding of Syntactic Trees for Machine Comprehension

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 815–824 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Structural Embedding of Syntactic Trees for Machine Comprehension Rui Liu∗, Junjie Hu∗, Wei Wei∗, Zi Yang∗, Eric Nyberg School of Computer Science Carnegie Mellon University 5000 Forbes Ave, Pittsburgh PA 15213, USA {ruil, junjieh, weiwei, ziy, ehn}@cs.cmu.edu Abstract Deep neural networks for machine com- prehension typically utilizes only word or character embeddings without explic- itly taking advantage of structured linguis- tic information such as constituency trees and dependency trees. In this paper, we propose structural embedding of syntactic trees (SEST), an algorithm framework to utilize structured information and encode them into vector representations that can boost the performance of algorithms for the machine comprehension. We evaluate our approach using a state-of-the-art neu- ral attention model on the SQuAD dataset. Experimental results demonstrate that our model can accurately identify the syntactic boundaries of the sentences and extract an- swers that are syntactically coherent over the baseline methods. 1 Introduction Reading comprehension such as SQuAD (Ra- jpurkar et al., 2016) or NewsQA (Trischler et al., 2016) requires identifying a span from a given context, which is an extension to the traditional question answering task, aiming at responding questions posed by human with natural language (Nyberg et al., 2002; Ferrucci et al., 2010; Liu, 2017; Yang, 2017). Many works have been pro- posed to leverage deep neural networks for such question answering tasks, most of which involve learning the query-aware context representations (Dhingra et al., 2016; Seo et al., 2017; Wang and Jiang, 2016; Xiong et al., 2017). Although deep learning based methods demonstrated great poten- tials for question answering, none them take syn- tactic information of the sentences such as con- ∗Authors contributed equally to this work. stituency tree and dependency tree into considera- tion. Such techniques have been proven to be use- ful in many natural language understanding tasks in the past and illustrated noticeable improvements such as the work by (Rajpurkar et al., 2016). In this paper, we adopt similar ideas but apply them to a neural attention model for question answering. The constituency tree (Manning et al., 1999) of a sentence defines internal nodes and termi- nal nodes to represent phrase structure grammars and the actual words. Figure 1 illustrates the con- stituency tree of the sentence “the architect or en- gineer acts as the project coordinator”. Here, “the architect or engineer” and “the project coordina- tor” are labeled as noun phrases (“NP”), which is critical for answering the question below. Here, the question asks for the name of certain occu- pation that can be best answered using an noun phrase. Utilizing the know ledge of a constituency relations, we can reduce the size of the candidate space and help the algorithm to identify the correct answer. Whose role is to design the works, prepare the specifications and produce construction drawings, administer the contract, tender the works, and manage the works from inception to completion? …

Content truncated to 500 words