Joint Information Extraction and Reasoning: A Scalable Statistical Relational Learning Approach

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 355–364, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Joint Information Extraction and Reasoning: A Scalable Statistical Relational Learning Approach William Yang Wang Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213, USA yww@cs.cmu.edu William W. Cohen Machine Learning Department Carnegie Mellon University Pittsburgh, PA 15213, USA wcohen@cs.cmu.edu Abstract A standard pipeline for statistical rela- tional learning involves two steps: one first constructs the knowledge base (KB) from text, and then performs the learn- ing and reasoning tasks using probabilis- tic first-order logics. However, a key is- sue is that information extraction (IE) er- rors from text affect the quality of the KB, and propagate to the reasoning task. In this paper, we propose a statistical rela- tional learning model for joint information extraction and reasoning. More specifi- cally, we incorporate context-based entity extraction with structure learning (SL) in a scalable probabilistic logic framework. We then propose a latent context inven- tion (LCI) approach to improve the per- formance. In experiments, we show that our approach outperforms state-of-the-art baselines over three real-world Wikipedia datasets from multiple domains; that joint learning and inference for IE and SL sig- nificantly improve both tasks; that latent context invention further improves the re- sults. 1 Introduction Information extraction (IE) is often an early stage in a pipeline that contains non-trivial downstream tasks, such as question answering (Moll´a et al., 2006), machine translation (Babych and Hartley, 2003), or other applications (Wang and Hua, 2014; Li et al., 2014). Knowledge bases (KBs) populated by IE techniques have also been used as an input to systems that learn rules allowing further infer- ences to be drawn from the KB (Lao et al., 2011), a task sometimes called KB completion (Socher et al., 2013; Wang et al., 2014; West et al., 2014). Pipelines of this sort frequently suffer from error cascades, which reduces performance of the full system1. In this paper, we address this issue, and pro- pose a joint model system for IE and KB com- pletion in a statistical relational learning (SRL) setting (Sutton and McCallum, 2006; Getoor and Taskar, 2007). In particular, we outline a system which takes as input a partially-populated KB and a set of relation mentions in context, and jointly learns: 1) how to extract new KB facts from the relation mentions, and; 2) a set of logical rules that allow one to infer new KB facts. Evaluation of the KB facts inferred by the joint system shows that the joint model outperforms its individual com- ponents. We also introduce a novel extension of this model called Latent Context Invention (LCI), which associates latent states with context features for the IE component of the model. We show that LCI further improves performance, leading to a substantial improvement over prior state-of-the-art methods for joint relation-learning and IE. To summarize our contributions: • We present a joint model for …

Content truncated to 500 words