Tracing Origins: Coreference-aware Machine Reading Comprehension

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1281 - 1292 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Tracing Origins: Coreference-aware Machine Reading Comprehension Baorong Huang1,#, Zhuosheng Zhang2,3,#, Hai Zhao2,3,∗ 1Institute of Corpus Studies and Applications, Shanghai International Studies University 2Department of Computer Science and Engineering, Shanghai Jiao Tong University 3Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University 0214101730@shisu.edu.cn, zhangzs@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract Machine reading comprehension is a heavily- studied research and test field for evaluating new pre-trained language models (PrLMs) and fine-tuning strategies, and recent studies have enriched the pre-trained language models with syntactic, semantic and other linguistic information to improve the performance of the models. In this paper, we imitate the human reading process in connecting the anaphoric expressions and explicitly leverage the coreference information of the entities to enhance the word embeddings from the pre- trained language model, in order to highlight the coreference mentions of the entities that must be identified for coreference-intensive question answering in QUOREF, a relatively new dataset that is specifically designed to evaluate the coreference-related performance of a model. We use two strategies to fine- tune a pre-trained language model, namely, placing an additional encoder layer after a pre-trained language model to focus on the coreference mentions or constructing a rela- tional graph convolutional network to model the coreference relations. We demonstrate that the explicit incorporation of coreference information in the fine-tuning stage performs better than the incorporation of the coreference information in pre-training a language model. 1 Introduction Machine reading comprehension (MRC), a task that automatically identifies one or multiple words from a given passage as the context to answer a specific question for that passage, is widely used in information retrieving, search engines, and dialog systems. Several datasets on MRC that limit the answer to one single word or multiple words from the passage are introduced, including TREC ∗Corresponding author. # Equal contribution. This work was supported in part by the Key Projects of National Natural Science Foundation of China under Grants U1836222 and 61733011. Context: Frankie Bono, a mentally disturbed hitman from Cleveland, comes back to his hometown in New York City during Christmas week to kill a middle-management mobster, Troiano. ...First he follows his target to select the best possible location, but opts to wait until Troiano isn’t being accompanied by his bodyguards. ... Losing his nerve, Frankie calls up his employers to tell them he wants to quit the job. Unsympathetic, the supervisor tells him he has until New Year’s Eve to perform the hit. Question: What is the first name of the person who has until New Year’s Eve to perform a hit? Answer: he - >Frankie Question: What is the first name of the person who follows their target to select the best possible location? Answer: he ->Frankie Table 1: An example from QUOREF: coreference resolution is required to extract the correct answer. We highlight the supporting text …

Content truncated to 500 words