Lyb3b at SemEval-2018 Task 11: Machine Comprehension Task using Deep Learning Models

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1073–1077 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics ZMU at SemEval-2018 Task 11: Machine Comprehension Task using Deep Learning Models Yongbin Li1,2, Xiaobing Zhou1,∗ 1Yunnan University, Kunming, Yunnan, P.R. China 2Zunyi Medical University, Zunyi, Guizhou, P.R. China ∗ Corresponding author, zhouxb.cn@gmail.com Abstract Machine Comprehension of text is a typical Natural Language Processing task which re- mains an elusive challenge. This paper is to solve the task 11 of SemEval-2018, Machine Comprehension using Commonsense Knowl- edge task. We use deep learning model to solve the problem. We build distributed word embedding of text, question and answering re- spectively instead of manually extracting fea- tures by linguistic tools. Meanwhile, we use a series of frameworks such as CNN model, LSTM model, LSTM with attention model and biLSTM with attention model for processing word vector. Experiments demonstrate the su- perior performance of biLSTM with attention framework compared to other models. We also delete high frequency words and combine word vector and data augmentation methods, achieved a certain effect. The approach we proposed rank 6th in official results, with ac- curacy rate of 0.7437 in test dataset. 1 Introduction Machine Comprehension of text is one of the important goals of natural language processing. The traditional approaches to machine reading and comprehension have been based on either hand en- gineered grammars (Riloff and Thelen, 2000), or information extraction methods of detecting pred- icate argument triples that can later be queried as a relational database (Poon et al., 2010). These methods show effectiveness, but they rely on fea- ture extraction and language tools. Recently, with the advances of neural networks, there have been great interests in building neural architectures for various NLP task, including several pieces of work on machine comprehension (Hermann et al., 2015; Hill et al., 2015; Yin et al., 2016; Kadlec et al., 2016; Cui et al., 2016), which have gained signif- icant performance in machine comprehension do- main. We also adopt deep learning models to solve this task. The goal of Machine Comprehension using Commonsense Knowledge task is to choose a cor- rect answer in two candidates to the question based on the contents of text. This task relates to how the inclusion of commonsense knowledge in the form of script knowledge would benefit machine comprehension systems, answering the questions requires knowledge beyond the facts mentioned in the text. We do not employ extra commonsense knowledge resources in the proposed approach, we assume that word vectors have contained some commonsense knowledge information, so we only use the deep learning model to solve this problem. In the train dataset provided by this task, there are 1432 instances, each instance contains a text and several questions, and each question is asso- ciated with a set of two answers which are short and limited to a few words. The texts used in this task cover more than 100 everyday scenarios, hence include a wide variety of human activities. Therefore, each …

Content truncated to 500 words