Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension

Proceedings of the 28th International Conference on Computational Linguistics, pages 2635–2641 Barcelona, Spain (Online), December 8-13, 2020 2635 Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension Shaoru Guo1, Yong Guan1, Ru Li 1,2∗, Xiaoli Li3, Hongye Tan1,2 1. School of Computer and Information Technology, Shanxi University, Taiyuan, China 2. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan, China 3. Institute for Infocomm Research, A*Star, Singapore {guoshaoru0928,guanyong0130}@163.com {liru,tanhongye}@sxu.edu.cn,xlli@i2r.a-star.edu.sg Abstract Machine reading comprehension (MRC) is one of the most critical yet challenging tasks in nat- ural language understanding(NLU), where both syntax and semantics information of text are essential components for text understanding. It is surprising that jointly considering syntax and semantics in neural networks was never formally reported in literature. This paper makes the first attempt by proposing a novel Syntax and Frame Semantics model for Machine Reading Comprehension (SS-MRC), which takes full advantage of syntax and frame semantics to get richer text representation. Our extensive experimental results demonstrate that SS-MRC per- forms better than ten state-of-the-art technologies on machine reading comprehension task. 1 Introduction Machine Reading Comprehension (MRC) requires machines to read and understand a text passage, and answer relevant questions about it, where MRC systems must have the ability to infer the meanings of underlying natural language, where both syntax and semantics are critical for text understanding. Traditional MRC methods are feature-based approaches, where they first manually generate syntactic or semantic features and subsequently apply a standard machine learning model to identify a best answer. For instance, Sachan and Xing (2016) use the Abstract Meaning Representation (AMR) formalism in a max-margin framework, which only focuses on semantic information. In addition, there are studies that take both syntax and semantics into account for MRC (Wang et al., 2015; Li et al., 2018). However, these methods heavily rely on manually defined features and are difficult to generalize to other tasks. Recently, as large QA datasets become available, Neural-based methods have been proposed for MRC, where even without time consuming feature engineering, they can still achieve favorable results. Espe- cially significant progress has been achieved on MRC tasks by fine-tuning a pre-trained general purpose language model (Devlin et al., 2018; Radford et al., 2018). Despite the success of those neural models, a number of studies have found there is a huge gap between MRC deep learning models and human beings (Wang and Jiang, 2019), as they might not really understand the natural language text (Mudrakarta et al., 2018). As such, some works employ semantic knowledge in neural network to facilitate sentence modelling (Guo et al., 2020; Zhang et al., 2018a; Zhang et al., 2018b). However, no work has been focused on integrating syntax and semantics into neural network for MRC. Note that FrameNet (Fillmore, 1976; Baker et al., 1998), as a widely adopted knowledge base, provides rich schematic scenario representation that could be potentially leveraged to better understand sentences. In this paper, we proposed a Syntax and Frame Semantics model for …

Content truncated to 500 words