HRCA+: Advanced Multiple-choice Machine Reading Comprehension Method

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6059–6068 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6059 HRCA+: Advanced Multiple-choice Machine Reading Comprehension Method Yuxiang Zhang, Hayato Yamana Waseda University 3 Chome-4-1 Okubo, Shinjuku, Tokyo 169-8555, Japan joel0495@asagi.waseda.jp, yamana@yama.info.waseda.ac.jp Abstract Multiple-choice question answering (MCQA) for machine reading comprehension (MRC) is challenging. It requires a model to select a correct answer from several candidate options related to text passages or dialogue. To select the correct answer, such models must have the ability to understand natural languages, comprehend textual representations, and infer the relationship between candidate options, questions, and passages. Previous models calculated representations between passages and question-option pairs separately, thereby ignoring the effect of other relation-pairs. In this study, we propose a human reading comprehension attention (HRCA) model and a passage-question-option (PQO) matrix-guided HRCA model called HRCA+ to increase accuracy. The HRCA model updates the information learned from the previous relation-pair to the next relation-pair. HRCA+ utilizes the textual information and the interior relationship between every two parts in a passage, a question, and the corresponding candidate options. Our proposed method outperforms other state-of-the-art methods. On the Semeval-2018 Task 11 dataset, our proposed method improved accuracy levels from 95.8% to 97.2%, and on the DREAM dataset, it improved accuracy levels from 90.4% to 91.6% without extra training data, from 91.8% to 92.6% with extra training data. Keywords: natural language processing, machine reading comprehension, multiple-choice question answering 1. Introduction Machine reading comprehension (MRC) is a challeng- ing task that involves training a model to comprehend the meaning of documents written in natural languages. MRC has attracted significant attention in the field of artificial intelligence, and it was developed to measure how deeply a machine understands context (Liu et al., 2019a). Note that MRC requires a model, especially a supervised learning model, to answer questions based on a specific context. Researchers are expected to train a model to orientate a passage and question pair to- wards the corresponding answer. MRC tasks are classi- fied into four types (Chen, 2018): cloze test, multiple- choice, span extraction, and free answering. In this study, we tackle the multiple-choice question answering task. Multiple-choice tasks, which are com- monly used in language proficiency exams, require se- lecting one correct answer among multiple candidate options according to a passage. Because the ranges of questions and options are not limited in a passage, some questions require inference combined with com- monsense, and this cannot be achieved only using an information retrieval system or through pattern match- ing. Therefore, pre-trained language models (PrLMs) for understanding a passage, together with matching networks for capturing the relationship between a pas- sage, a question, and the candidate options, are helpful in and crucial to the tackling of multiple-choice tasks. MMM (Jin et al., 2020), DCMN+ (Zhang et al., 2020), and DUMA (Zhu et al., 2020) are three state-of-the- art methods that adopt PrLMs as the encoders of their models. Although all these methods …

Content truncated to 500 words