Multi-Perspective Context Aggregation for Semi-supervised Cloze-style Reading Comprehension

Proceedings of the 27th International Conference on Computational Linguistics, pages 857–867 Santa Fe, New Mexico, USA, August 20-26, 2018. 857 Multi-Perspective Context Aggregation for Semi-supervised Cloze-style Reading Comprehension Liang Wang1, Sujian Li2, Wei Zhao1, Kewei Shen1, Meng Sun1, Ruoyu Jia1, Jingming Liu1 1Yuanfudao Research, Beijing, China 2Key Laboratory of Computational Linguistics, Peking University, MOE, China {wangliang01,zhaowei01,shenkw,sunmeng,jiary,liujm}@fenbi.com lisujian@pku.edu.cn Abstract Cloze-style reading comprehension has been a popular task for measuring the progress of natural language understanding in recent years. In this paper, we design a novel multi-perspective frame- work, which can be seen as the joint training of heterogeneous experts and aggregate context information from different perspectives. Each perspective is modeled by a simple aggregation module. The outputs of multiple aggregation modules are fed into a one-timestep pointer net- work to get the final answer. At the same time, to tackle the problem of insufficient labeled data, we propose an efficient sampling mechanism to automatically generate more training examples by matching the distribution of candidates between labeled and unlabeled data. We conduct our experiments on a recently released cloze-test dataset CLOTH (Xie et al., 2017), which consists of nearly 100k questions designed by professional teachers. Results show that our method achieves new state-of-the-art performance over previous strong baselines. 1 Introduction Reading comprehension is a challenging task which requires the deep understanding of natural language. Cloze test is a particular form of reading comprehension: given a passage with blanks, an examinee is required to fill in the missing word (or phrase) that best fits the context surrounding the blank. Recently, cloze-style reading comprehension has drawn growing interests from NLP research communities, since such a task meets the practical requirements and is relatively easy to design. The research of cloze-style reading comprehension is first advanced by two large-scale corpora: the CNN/Daily Mail (Hermann et al., 2015) and CBT (Hill et al., 2015) datasets, which are automatically constructed by randomly or periodically deleting a word from original passage. Though the automatically generated datasets usually consist of a large quantity of labeled data and make it possible to train large neural network models, they are in nature far away from real-world language understanding problems and have serious ambiguity issues (Chen et al., 2016). As a result, the state-of-the-art system of cloze test almost reaches the performance ceiling and loses its improvement direction due to the limitation of the corpus (Chen et al., 2016). In such situation, Xie et al. (2017) argues that it is a more reliable means to assess language proficiency with carefully designed questions by professional teachers, and releases a novel corpus CLOTH. The CLOTH dataset brings the new challenge of exploring a comprehensive evaluation of language proficiency and specifically divides the questions into several types including matching, reasoning and grammar etc. Table 1 shows several example questions from CLOTH. From experiments by Xie et al. (2017), we can see that the Stanford attention reader (Chen et al., 2016) of having the near state-of-the-art performance (with an accuracy of about 0.74) on CNN/Daily Mail …

Content truncated to 500 words