Natural Solution to FraCaS Entailment Problems

Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics (*SEM 2016), pages 64–74, Berlin, Germany, August 11-12, 2016. Natural Solution to FraCaS Entailment Problems Lasha Abzianidze TiLPS, Tilburg University, the Netherlands L.Abzianidze@uvt.nl Abstract Reasoning over several premises is not a common feature of RTE systems as it usually requires deep semantic analysis. On the other hand, FraCaS is a collec- tion of entailment problems consisting of multiple premises and covering semanti- cally challenging phenomena. We employ the tableau theorem prover for natural lan- guage to solve the FraCaS problems in a natural way. The expressiveness of a type theory, the transparency of natural logic and the schematic nature of tableau infer- ence rules make it easy to model chal- lenging semantic phenomena. The effi- ciency of theorem proving also becomes challenging when reasoning over several premises. After adapting to the dataset, the prover demonstrates state-of-the-art com- petence over certain sections of FraCaS. 1 Introduction Understanding and automatically processing the natural language semantics is a central task for computational linguistics and its related fields. At the same time, inference tasks are regarded as the best way of testing an NLP systems’s semantic ca- pacity (Cooper et al., 1996, p. 63). Following this view, recognizing textual entailment (RTE) chal- lenges (Dagan et al., 2005) were regularly held which evaluate the RTE systems based on the RTE dataset. The RTE data represents a set of text- hypotheses pairs that are human annotated on the inference relations: entailment, contradiction and neutral. Hence it attempts to evaluate the systems on human reasoning. In general, the RTE datasets are created semi-automatically and are often mo- tivated by the scenarios found in the applications like question answering, relation extraction, infor- mation retrieval and summarization (Dagan et al., 2005; Dagan et al., 2013). On the other hand, the semanticists are busy designing theories that account for the valid logical relations over nat- ural language sentences. These theories usually model reasoning that depends on certain seman- tic phenomena, e.g., Booleans, quantifiers, events, attitudes, intensionality, monotonicity, etc. These types of reasoning are weak points of RTE systems as the above mentioned semantic phenomena are underrepresented in the RTE datasets. In order to test and train the weak points of an RTE system, we choose the FraCaS dataset (Cooper et al., 1996). The set contains complex entailment problems covering various challeng- ing semantic phenomena which are still not fully mastered by RTE systems. Moreover, unlike the standard RTE datasets, FraCaS also allows multi- premised problems. To account for these com- plex entailment problems, we employ the theorem prover for higher-order logic (Abzianidze, 2015a), which represents the version of formal logic mo- tivated by natural logic (Lakoff, 1970; Van Ben- them, 1986). Though such expressive logics usu- ally come with the inefficient decision procedures, the prover maintains efficiency by using the infer- ence rules that are specially tailored for the reason- ing in natural language. We introduce new rules for the prover in light of the FraCaS problems and test …

Content truncated to 500 words