Tables as Semi-structured Knowledge for Question Answering

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 474–483, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Tables as Semi-structured Knowledge for Question Answering Sujay Kumar Jauhar Carnegie Mellon University Pittsburgh, PA, USA sjauhar@cs.cmu.edu Peter D. Turney Allen Institute for Artificial Intelligence Seattle, WA, USA petert@allenai.org Eduard Hovy Carnegie Mellon University Pittsburgh, PA, USA hovy@cs.cmu.edu Abstract Question answering requires access to a knowledge base to check facts and rea- son about information. Knowledge in the form of natural language text is easy to ac- quire, but difficult for automated reason- ing. Highly-structured knowledge bases can facilitate reasoning, but are difficult to acquire. In this paper we explore tables as a semi-structured formalism that pro- vides a balanced compromise to this trade- off. We first use the structure of tables to guide the construction of a dataset of over 9000 multiple-choice questions with rich alignment annotations, easily and ef- ficiently via crowd-sourcing. We then use this annotated data to train a semi- structured feature-driven model for ques- tion answering that uses tables as a knowl- edge base. In benchmark evaluations, we significantly outperform both a strong un- structured retrieval baseline and a highly- structured Markov Logic Network model. 1 Introduction Question answering (QA) has emerged as a prac- tical research problem for pushing the boundaries of artificial intelligence (AI). Dedicated projects and open challenges to the research community in- clude examples such as Facebook AI Research’s challenge problems for AI-complete QA (Weston et al., 2015) and the Allen Institute for AI’s (AI2) Aristo project (Clark, 2015) along with its recently completed Kaggle competition1. The reason for this emergence is the diversity of core language and reasoning problems that a complex, integrated 1https://www.kaggle.com/c/ the-allen-ai-science-challenge task like QA exposes: information extraction (Sri- hari and Li, 1999), semantic modelling (Shen and Lapata, 2007; Narayanan and Harabagiu, 2004), logic and reasoning (Moldovan et al., 2003), and inference (Lin and Pantel, 2001). Complex tasks such as QA require some form of knowledge base to store facts about the world and reason over them. By knowledge base, we mean any form of knowledge: structured (e.g., ta- bles, ontologies, rules) or unstructured (e.g., nat- ural language text). For QA, knowledge has been harvested and used in a number of different modes and formalisms: large-scale extracted and curated knowledge bases (Fader et al., 2014), structured models such as Markov Logic Networks (Khot et al., 2015), and simple text corpora in information retrieval approaches (Tellex et al., 2003). There is, however, a fundamental trade-off in the structure and regularity of a formalism and its ability to be curated, modelled or reasoned with easily. For example, simple text corpora contain no structure, and are therefore hard to reason with in a principled manner. Nevertheless, they are eas- ily and abundantly available. In contrast, Markov Logic Networks come with a wealth of theoretical knowledge connected with their usage in princi- pled inference. However, they are difficult to in- duce automatically from text or to build …

Content truncated to 500 words