TruthTeller: Annotating Predicate Truth

Proceedings of NAACL-HLT 2013, pages 752–757, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics TruthTeller: Annotating Predicate Truth Amnon Lotan Department of Linguistics Tel Aviv University amnonlot@post.tau.ac.il Asher Stern and Ido Dagan Department of Computer Science Bar Ilan University astern7@gmail.com dagan@cs.biu.ac.il Abstract We propose a novel semantic anno- tation type of assigning truth values to predicate occurrences, and present TruthTeller, a standalone publicly- available tool that produces such annota- tions. TruthTeller integrates a range of semantic phenomena, such as nega- tion, modality, presupposition, implicativ- ity, and more, which were dealt only partly in previous works. Empirical evaluations against human annotations show satisfac- tory results and suggest the usefulness of this new type of tool for NLP. 1 Introduction In a text, the action or relation denoted by ev- ery predicate can be seen as being either pos- itively or negatively inferred from its sentence, or otherwise having an unknown truth status. Only in (3) below can we infer that Gal sold her shop, hence the positive truth value of the predicate sell, while according to (2) and (4) Gal did not sell it, hence the negative truth values, and in (1) we do not know if she sold it or not (the notations pt+, pt- and pt? denote truth states, defined in Subsection 2.3). Identifying these predicate truth values is an important sub- task within many semantic processing scenarios, including various applications such as Question Answering (QA), Information Extraction (IE), paraphrasing and summarization. The follow- ing examples illustrate the phenomenon: (1) Gal made an attempt pt+ to sell pt? her shop. (2) Gal did not try pt− to sell pt− her shop af- ter hearing pt+ the offers. (3) Maybe Gal wasn’t smart pt? to sell pt+ her shop. (4) Gal wasn’t smart pt− enough to sell pt− the shop that she had bought pt+. Previous works addressed specific aspects of the truth detection problem: Nairn et al. (2006), and later MacCartney & Manning (2007; 2009), were the first to build paraphrasing and inference systems that combine negation (see try in (2)), modality (smart in (3)) and “natural logic”, a recursive truth value calculus (sell in (1-3)); recently, Mausam et al. (2012) built an open IE system that identifies granulated vari- ants of modality and conditions on predicates (smart in (3)); and Kiparsky & Kiparsky (1970) and Karttunen (1971; 2012) laid the ground work for factive and implicative entailment cal- culus (sell in (1-4)), as well as many generic constructions of presupposition (hearing in (2) is presupposed because it heads an adverbial clause and bought in (4) heads a finite relative clause), which, to our knowledge, have not yet been implemented computationally. Notice in the examples that presuppositions persist under negation, in questions and if-clauses, while en- tailments do not. In addition, there is a growing research line of negation and modality detection. See, for example, Morante & Daelemans (2012). 752 We present TruthTeller1, a novel algo- rithm and system that identifies the truth value of each predicate …

Content truncated to 500 words