Consensus Attention-based Neural Networks for Chinese Reading Comprehension

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1777–1786, Osaka, Japan, December 11-17 2016. Consensus Attention-based Neural Networks for Chinese Reading Comprehension Yiming Cui†∗, Ting Liu‡, Zhipeng Chen†, Shijin Wang† and Guoping Hu† †iFLYTEK Research, Beijing, China ‡Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, China †{ymcui,zpchen,sjwang3,gphu}@iflytek.com ‡tliu@ir.hit.edu.cn Abstract Reading comprehension has embraced a booming in recent NLP research. Several institutes have released the Cloze-style reading comprehension data, and these have greatly accelerated the research of machine comprehension. In this work, we firstly present Chinese reading compre- hension datasets, which consist of People Daily news dataset and Children’s Fairy Tale (CFT) dataset. Also, we propose a consensus attention-based neural network architecture to tackle the Cloze-style reading comprehension problem, which aims to induce a consensus attention over every words in the query. Experimental results show that the proposed neural network signif- icantly outperforms the state-of-the-art baselines in several public datasets. Furthermore, we setup a baseline for Chinese reading comprehension task, and hopefully this would speed up the process for future research. 1 Introduction The ultimate goal of machine intelligence is to read and comprehend human languages. Among vari- ous machine comprehension tasks, in recent research, the Cloze-style reading comprehension task has attracted lots of researchers. The Cloze-style reading comprehension problem (Taylor, 1953) aims to comprehend the given context or document, and then answer the questions based on the nature of the document, while the answer is a single word in the document. Thus, the Cloze-style reading comprehen- sion can be described as a triple: ⟨D, Q, A⟩ where D is the document, Q is the query and A is the answer to the query. By adopting attention-based neural network approaches (Bahdanau et al., 2014), the machine is able to learn the relationships between document, query and answer. But, as is known to all, the neural network based approaches need large-scale training data to train a reliable model for predictions. Hermann et al. (2015) published the CNN/Daily Mail news corpus for Cloze-style reading comprehensions, where the content is formed by the news articles and its summarization. Also, Hill et al. (2015) released the Chil- dren’s Book Test (CBT) corpus for further research, where the training samples are generated through automatic approaches. As we can see that, automatically generating large-scale training data for neural network training is essential for reading comprehension. Furthermore, more difficult problems, such as reasoning or summarization of context, need much more data to learn the higher-level interactions. Though we have seen many improvements on these public datasets, some researchers suggested that these dataset requires less high-level inference than expected (Chen et al., 2016). Furthermore, the public datasets are all automatically generated, which indicate that the pattern in training and testing phase are nearly the same, and this will be easier for the machine to learn these patterns. In this paper, we will release Chinese reading comprehension datasets, including People Daily news datasets and Children’s Fairy Tale datasets. As …

Content truncated to 500 words