Solving Verbal Questions in IQ Test by Knowledge-Powered Word Embedding

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 541–550, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Solving Verbal Questions in IQ Test by Knowledge-Powered Word Embedding Huazheng Wang University of Virginia hw7ww@virginia.edu Fei Tian Microsoft Research fetia@microsoft.com Bin Gao Microsoft bingao@microsoft.com Chengjieren Zhu University of California, San Diego chz191@ucsd.edu Jiang Bian Yidian Inc. jiang.bian.prc@gmail.com Tie-Yan Liu Microsoft Research tyliu@microsoft.com Abstract Verbal comprehension questions appear very frequently in Intelligence Quotient (IQ) tests, which measure human’s verbal ability includ- ing the understanding of the words with mul- tiple senses, the synonyms and antonyms, and the analogies among words. In this work, we explore whether such tests can be solved automatically by the deep learning technolo- gies for text data. We found that the task was quite challenging, and simply applying existing technologies like word embedding could not achieve a good performance, due to the multiple senses of words and the com- plex relations among words. To tackle these challenges, we propose a novel framework to automatically solve the verbal IQ questions by leveraging improved word embedding by jointly considering the multi-sense nature of words and the relational information among words. Experimental results have shown that the proposed framework can not only outper- form existing methods for solving verbal com- prehension questions but also exceed the aver- age performance of the Amazon Mechanical Turk workers involved in the study. 1 Introduction The Intelligence Quotient (IQ) test (Stern, 1914) is a test of intelligence designed to formally study the success of an individual in adapting to a spe- cific situation under certain conditions. Common IQ tests measure various types of abilities such as verbal, mathematical, logical, and reasoning skills. These tests have been widely used in the study of psychology, education, and career development. In the community of artificial intelligence, agents have been invented to fulfill many interesting and chal- lenging tasks like face recognition, speech recogni- tion, handwriting recognition, and question answer- ing. However, as far as we know, there are very lim- ited studies of developing an agent to solve IQ tests, which in some sense is more challenging, since even common human beings do not always succeed on such tests. Considering that IQ test scores have been widely considered as a measure of intelligence, we think it is worth further investigating whether we can develop an agent that can solve IQ test questions. The commonly used IQ tests contain several types of questions like verbal, mathematical, logical, and picture questions, among which a large proportion (near 40%) are verbal questions (Carter, 2005). The recent progress on deep learning for natural lan- guage processing (NLP), such as word embedding technologies, has advanced the ability of machines (or AI agents) to understand the meaning of words and the relations among words. This inspires us to solve the verbal questions in IQ tests by lever- aging the word embedding technologies. However, our attempts show that a straightforward applica- tion of word embedding does not result in …

Content truncated to 500 words