Large-Scale Acquisition of Entailment Pattern Pairs by Exploiting Transitivity

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1649–1655, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Large-Scale Acquisition of Entailment Pattern Pairs by Exploiting Transitivity Julien Kloetzer∗ Kentaro Torisawa‡ Chikara Hashimoto§ Jong-Hoon Oh¶ Information Analysis Laboratory, National Institute of Information and Communications Technology (NICT), Kyoto, Japan {∗ julien, ‡ torisawa, § ch, ¶rovellia}@nict.go.jp Abstract We propose a novel method for acquiring entailment pairs of binary patterns on a large-scale. This method exploits the tran- sitivity of entailment and a self-training scheme to improve the performance of an already strong supervised classifier for en- tailment, and unlike previous methods that exploit transitivity, it works on a large- scale. With it we acquired 138.1 million pattern pairs with 70% precision with such non-trivial lexical substitution as “use Y to distribute X”→“X is available on Y” whose extraction is considered difficult. This represents 50.4 million more pattern pairs (a 57.5% increase) than what our supervised baseline extracted at the same precision. 1 Introduction Recognizing textual entailment (Geffet and Da- gan, 2005; Androutsopoulos and Malakasiotis, 2009; Zanzotto et al., 2009; Berant et al., 2011) is an important task for many NLP applications, such as relation extraction (Romano et al., 2006) or question-answering (Harabagiu and Hickl, 2006). Text L entails text R if the information written in the latter can be deduced from the information written in the former. As building blocks to rec- ognize entailment relations between texts, numer- ous works have focused on recognizing entailment relations between patterns, such as “grew up in X”→“lived in X” or “X grew up in Y”→“X lived in Y” (Lin and Pantel, 2001; Weeds and Weir, 2003a; Hashimoto et al., 2009; Berant et al., 2011; Kloet- zer et al., 2013b). We propose in this paper a method for ac- quiring on a very large-scale, entailment pairs of Quantity of training data Average precision Baseline plus 5,000 training data samples 49.0% Baseline: 83,800 training data samples 48.8% Baseline minus 10,000 training data samples 48.5% Baseline minus 20,000 training data samples 47.8% Table 1: Average precision for baseline method with various amounts of training data such class-dependent binary patterns as “under- went Xexam on Ydate”→“Xexam carried out on Ydate”. Our starting point is a supervised baseline trained with 83,800 manually labeled pattern pairs detailed in Kloetzer et al. (2013b). Its top 205 mil- lion output pairs have an estimated 80% precision, but this baseline’s performance is saturated. Ta- ble 1 shows the baseline’s average precision when varying its amount of hand-labeled training data. Since the average precision only improves slightly with additional training data, the investment in hand-labeling additional training data is difficult to justify. To improve our baseline further, we exploit the transitivity property of entailment to automatically generate new features for it. The entailment is transitive; if we detect that L entails C and C en- tails R, we can infer an entailment relation be- tween L and R even if no such relation was de- tected beforehand. Based …

Content truncated to 500 words