YNU_Deep at SemEval-2018 Task 11: An Ensemble of Attention-based BiLSTM Models for Machine Comprehension

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1043–1047 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics YNU Deep at SemEval-2018 Task 11: An Ensemble of Attention-based BiLSTM Models for Machine Comprehension Peng Ding, Xiaobing Zhou∗ School of Information Science and Engineering Yunnan University, Yunnan, P.R. China ∗Corresponding author, zhouxb.cn@gmail.com Abstract This paper reports our submission to task 11 (Machine Comprehension using Common- sense Knowledge) in SemEval 2018. We firstly use GloVe to learn the distributed rep- resentations automatically from the instance, question and answer triples. Then an attention- based Bidirectional LSTM (BiLSTM) model is used to encode the triples. We also perform a simple ensemble method to improve the ef- fectiveness of our model. The system we de- veloped obtains an encouraging result on this task. It achieves the accuracy 0.7472 on the test set. We rank 5th according to the official ranking. 1 Introduction Machine comprehension of text is one of the ulti- mate goals of natural language processing. The machine comprehension problem can be formu- lated as follows: Given an instance i, a question q and an answer candidate pool {a1, a2, . . ., as}, the aim is to search for the best answer candidate ak, where 1 ≤ k ≤ s. The major challenge of this task is that the words in the answer do not neces- sarily appear in the instance. In recent years, deep learning models are widely used in the field of NLP, such as semantic analysis (Tang et al., 2015), machine translation (Bahdanau et al., 2014) and text summarization (Rush et al., 2015). (Bahdanau et al., 2014) also introduced the attention mechanism into NLP task for the first time. This attention-based model yielded state- of-the-art performance on the machine translation task. (Hermann et al., 2015) built a supervised reading comprehension data set, the CNN/Daily Mail data sets1. They also presented Attentive Reader for machine comprehension, which allows a model to focus on the aspects of an instance that 1http://www.github.com/deepmind/rc-data/ can help to answer a question, and also allows us to visualize its inference process (Hermann et al., 2015). The key point of the attention-based mod- els is the design of attention function. Com- pared to Attentive Reader, Attention Sum Reader (Kadlec et al., 2016) used the dot products instead of a tanh layer to compute the attention between question and contextual embeddings. Stanford At- tention Reader (Chen et al., 2016) took a bilinear term as the attention function and obtained state- of-the-art results on the CNN/Daily Mail data sets. The reasoning process was implemented in some models for machine comprehension. Mem- ory Networks (Sukhbaatar et al., 2015) was the first model to propose reasoning process, which had important influence on other follow-up mod- els. Compared to the traditional attention model, Memory Networks additionally uses a function t that constantly updates the representation of the instance and the question so as to realize the rea- soning process. (Tseng et al., 2016) proposed an attention-based multi-hop recurrent …

Content truncated to 500 words