Bi-directional CognitiveThinking Network for Machine Reading Comprehension

Proceedings of the 28th International Conference on Computational Linguistics, pages 2613–2623 Barcelona, Spain (Online), December 8-13, 2020 2613 Bi-directional Cognitive Thinking Network for Machine Reading Comprehension Wei Peng1,2, Yue Hu1,2∗, Luxi Xing1,2 Yuqiang Xie1,2, Jing Yu1,2 ,Yajing Sun1,2, Xiangpeng Wei1,2 1Institute of Information Engineering, Chinese Academy of Sciences, China 2School of Cyber Security, University of Chinese Academy of Sciences, China {pengwei,huyue,xingluxi,xieyuqiang}@iie.ac.cn {yujing,sunyajing,weixiangpeng}@iie.ac.cn Abstract We propose a novel Bi-directional Cognitive Knowledge Framework (BCKF) for reading com- prehension from the perspective of complementary learning systems theory. It aims to sim- ulate two ways of thinking in the brain to answer questions, including reverse thinking and inertial thinking. To validate the effectiveness of our framework, we design a corresponding Bi-directional Cognitive Thinking Network (BCTN) to encode the passage and generate a ques- tion (answer) given an answer (question) and decouple the bi-directional knowledge. The model has the ability to reverse reasoning questions which can assist inertial thinking to generate more accurate answers. Competitive improvement is observed in DuReader dataset, confirming our hypothesis that bi-directional knowledge helps the QA task. The novel framework shows an interesting perspective on machine reading comprehension and cognitive science. 1 Introduction Machine Reading Comprehension (MRC) has made significant strides with an array of neural models rapidly approaching human parity on some benchmarks such as SQuAD (Rajpurkar et al., 2016). How- ever, existing methods are still in their infancy at the level of cognitive intelligence. Recently, brain science and psychology provide an important basis for the development of brain-like computing and the simulation of human perception, thinking, understanding, and reasoning abilities (Sharp et al., 2017). Thinking is the generalization and indirect reflection of the human brain on the nature, interrelation- ships and internal regularities of objective things (Wing, 2006). Two types of thinking are complementary in psychology: inertial thinking — from a previous to a subsequent stimulus — and reverse thinking — from a subsequent to a previous stimulus (Krutetskii, 1976). Inertial thinking (Li and Wen, 2018) is a conventional way of thinking, which thinks and solves problems from the previous ideas. Reverse think- ing (Krutetskii, 1976) is a creative way of thinking, opposite to the inertial thinking. Specifically, in the MRC task, the two types of thinking can be regarded as a process which reasons from questions (an- swers) to answers (questions). For example, as shown in Fig. 1, we can get the answer easily by locating the entities pregnant wowen and loquat. Contrarily, the generative question, which can be reasoned by reading the answer and passage, describes two aspects, including can pregnant women eat loquat and what is the benefit to eat loquat for pregnant women. We hope that this ability of reverse reasoning can improve performance on reading comprehension tasks. Previous methods (Wang and Jiang, 2017; Seo et al., 2017; Tan et al., 2017; Nishida et al., 2019) only consider a obverse logical relationship, which is based on the given question and the passage. They ignore the reverse relationship between the given passage and the answer. Although …

Content truncated to 500 words