NLP-IIS@UT at SemEval-2021 Task 4: Machine Reading Comprehension using the Long Document Transformer

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 205–210 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 205 NLP-IIS@UT at SemEval-2021 Task 4: Machine Reading Comprehension using the Long Document Transformer Hossein Basafa College of ECE University of Tehran hbasafa@ut.ac.ir Sajad Movahedi College of ECE University of Tehran s.movahedi@ut.ac.ir Ali Ebrahimi College of ECE University of Tehran ali96ebrahimi@ut.ac.ir Azadeh Shakery College of ECE University of Tehran shakery@ut.ac.ir Heshaam Faili College of ECE University of Tehran hfaili@ut.ac.ir Abstract This paper presents a technical report of our submission to the 4th task of SemEval-2021, titled: Reading Comprehension of Abstract Meaning. In this task, we want to predict the correct answer based on a question given a context. Usually, contexts are very lengthy and require a large receptive field from the model. Thus, common contextualized lan- guage models like BERT miss fine representa- tion and performance due to the limited capac- ity of the input tokens. To tackle this problem, we used the longformer model to better pro- cess the sequences. Furthermore, we utilized the method proposed in the longformer bench- mark on wikihop dataset which improved the accuracy on our task data from (23.01% and 22.95%) achieved by the baselines for subtask 1 and 2, respectively, to (70.30% and 64.38%). 1 Introduction Reading comprehension is the ability to understand a passage either by human or machine. One of the great benchmarks to evaluate this ability is to try to answer specific questions related to the passage (Rajpurkar et al., 2016). Generally, this problem can contain single or multiple documents as con- text (containing relevant information needed to un- derstand and answer the question), a question (a sentence with at least one asking parameter), and an answer (which is the parameter value of the question). In the Task of Reading Comprehension of Ab- stract Meaning (ReCAM), we have one passage as a context, one question and five candidate answers (Zheng et al., 2021). The goal is to identify the correct answer based on the context and the given question. You can see a sample of the data in Table 1. For each instance of the data, there is a passage, a question with a missing word that should be filled based on the passage, and five candidate answers to the question. Passage ... observers have even named it af- ter him, “Abenomics”. It is based on three key pillars - the “three arrows” of monetary policy, fiscal stimulus and structural reforms in order to ensure long-term sustainable growth in the world’s third-largest economy. In this weekend’s upper house elec- tions ... Question Abenomics: The @Placeholder and the risks Answer (A) chances (B) prospective (C) security (D) objectives (E) threats Table 1: An instance of the data. The task divides into two subtasks: impercepti- bility and non-specificity(Zheng et al., 2021). • imperceptibility: this level of abstract words refers to ideas and concepts that are distant from immediate perception; such as culture, economics, and politics. • non-specificity: In contrast …

Content truncated to 500 words