ELiRF-UPV at SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1034–1037 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics ELiRF-UPV at SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge Jos´e- ´Angel Gonz´alez, Llu´ıs-F. Hurtado, Encarna Segarra, Ferran Pla Universitat Polit`ecnica de Val`encia Cam´ı de Vera sn, 46022, Val`encia {jogonba2|lhurtado|esegarra|fpla}@dsic.upv.es Abstract This paper describes the participation of ELiRF-UPV team at task 11, Machine Com- prehension using Commonsense Knowledge, of SemEval-2018. Our approach is based on the use of word embeddings, NumberBatch Embeddings, and a Deep Learning architec- ture to find the best answer for the multiple- choice questions based on the narrative text. The results obtained are in line with those ob- tained by the other participants and they en- courage us to continue working on this prob- lem. 1 Introduction In the Machine Comprehension using Common- sense Knowledge task, systems must answer multiple-choice questions given narrative texts about everyday activities. In addition to what is mentioned in the text, a substantial number of questions require inference using script knowl- edge about different scenarios. In order to capture some script knowledge we decided to use a word representation based not only on distributional semantics word models but also on a knowledge graph, ConceptNet (Speer et al., 2016). ConceptNet is a knowledge graph that connects words and phrases of natural lan- guage with labeled edges. It is designed to rep- resent the general knowledge involved in under- standing language. ConceptNet could be used in combination with sources of distributional se- mantics, particularly the word2vec Google News skip-gram embeddings (Mikolov et al., 2013)) and GloVe 1.2 (Pennington et al., 2014), to produce new embeddings, NumberBatch embeddings, with state-of-the-art performance across many word- relatedness evaluations (Speer and Lowry-Duda, 2017). More specifically, NumberBatch is a list of se- mantic word vectors which contains a complex meaning of those terms, beyond containing only contextual information like other kinds of em- beddings based on distributional semantics e.g. Word2Vec or Glove. These embeddings are ob- tained through a combination of Word2Vec and Glove embeddings with knowledge extracted from ConceptNet by means of a technique known as retrofitting (Faruqui et al., 2014). In this work, we used word representations based on NumberBatch embeddings because these representations encode semantically rich informa- tion related to the commonsense. Moreover, in or- der to tackle this machine comprehension task, we used a Deep Learning architecture with new atten- tion mechanisms. The inclusion of these new at- tention mechanisms allow us to better capture the similarities among the elements of the input. The attention mechanisms we introduce in this work are suggested in the work (Seo et al., 2016), that obtained very competitive results in Question An- swering tasks. 2 Resources and Preprocess As we pointed in Section 1, NumberBatch embed- dings were used for the representation of words. These embedding are provided by ConceptNet 5, which was compiled by the Commonsense Com- puting Initiative. ConceptNet 5 is freely avail- able under the Creative Commons Attribution- ShareAlike license (CC BY …

Content truncated to 500 words