Creating Causal Embeddings for Question Answering with Minimal Supervision

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 138–148, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Creating Causal Embeddings for Question Answering with Minimal Supervision Rebecca Sharp, Mihai Surdeanu, Peter Jansen, Peter Clark, and Michael Hammond University of Arizona, Allen Institute for Artificial Intelligence {bsharp, msurdeanu, pajansen, hammond}@email.arizona.edu, PeterC@allenai.org Abstract A common model for question answering (QA) is that a good answer is one that is closely related to the question, where re- latedness is often determined using general- purpose lexical models such as word embed- dings. We argue that a better approach is to look for answers that are related to the ques- tion in a relevant way, according to the infor- mation need of the question, which may be determined through task-specific embeddings. With causality as a use case, we implement this insight in three steps. First, we generate causal embeddings cost-effectively by boot- strapping cause-effect pairs extracted from free text using a small set of seed patterns. Second, we train dedicated embeddings over this data, by using task-specific contexts, i.e., the context of a cause is its effect. Finally, we extend a state-of-the-art reranking approach for QA to incorporate these causal embed- dings. We evaluate the causal embedding models both directly with a casual implication task, and indirectly, in a downstream causal QA task using data from Yahoo! Answers. We show that explicitly modeling causality im- proves performance in both tasks. In the QA task our best model achieves 37.3% P@1, sig- nificantly outperforming a strong baseline by 7.7% (relative). 1 Introduction Question answering (QA), i.e., finding short answers to natural language questions, is one of the most im- portant but challenging tasks on the road towards natural language understanding (Etzioni, 2011). A common approach for QA is to prefer answers that are closely related to the question, where relatedness is often determined using lexical semantic models such as word embeddings (Yih et al., 2013; Jansen et al., 2014; Fried et al., 2015). While appealing for its robustness to natural language variation, this one- size-fits-all approach does not take into account the wide range of distinct question types that can appear in any given question set, and that are best addressed individually (Chu-Carroll et al., 2004; Ferrucci et al., 2010; Clark et al., 2013). Given the variety of question types, we suggest that a better approach is to look for answers that are related to the question through the appropriate re- lation, e.g., a causal question should have a cause- effect relation with its answer. If we adopt this view, and continue to work with embeddings as a mechanism for assessing relationship, this raises a key question: how do we train and use task-specific embeddings cost-effectively? Using causality as a use case, we answer this question with a framework for producing causal word embeddings with mini- mal supervision, and a demonstration that such task- specific embeddings significantly benefit causal QA. In particular, the contributions of this work are: (1) …

Content truncated to 500 words