A Study on Contextualized Language Modeling for Machine Reading Comprehension

1073

Authors

nan

Abstract

AbstractWith the recent breakthrough of deep learning technologies, research on machine reading comprehension (MRC) has attracted much attention and found its versatile applications in many use cases. MRC is an important natural language processing (NLP) task aiming to assess the ability of a machine to understand natural language expressions, which is typically operationalized by first asking questions based on a given text paragraph and then receiving machine-generated answers in accordance with the given context paragraph and questions. In this paper, we leverage two novel pretrained language models built on top of Bidirectional Encoder Representations from Transformers (BERT), namely BERT-wwm and MacBERT, to develop effective MRC methods. In addition, we also seek to investigate whether additional incorporation of the categorical information about a context paragraph can benefit MRC or not, which is achieved based on performing context paragraph clustering on the training dataset. On the other hand, an ensemble learning approach is proposed to harness the synergistic power of the aforementioned two BERT-based models so as to further promote MRC performance.

Keywords

nan

Full Content

�� The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) Taoyuan, Taiwan, October 15-16, 2021. The Association for Computational Linguistics and Chinese Language Processing 48 �� The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) Taoyuan, Taiwan, October 15-16, 2021. The Association for Computational Linguistics and Chinese Language Processing 49 �� The 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021) Taoyuan, Taiwan, October 15-16, 2021. The Association for Computational Linguistics and Chinese Language Processing 50 �� …

Content truncated to 500 words

Labeled Text

Publication Details

Paper ID:: 1073
Year:: 2021
Publication:: TIPSTER
Publication Type:: conf
DOI:: nan

Links

Paper URL PDF Publication URL