UTexas: Natural Language Semantics using Distributional Semantics and Probabilistic Logic

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 796–801, Dublin, Ireland, August 23-24, 2014. UTexas: Natural Language Semantics using Distributional Semantics and Probabilistic Logic Islam Beltagy∗, Stephen Roller∗, Gemma Boleda†, Katrin Erk†, Raymond J. Mooney∗ ∗ Department of Computer Science † Department of Linguistics The University of Texas at Austin {beltagy, roller, mooney}@cs.utexas.edu gemma.boleda@upf.edu, katrin.erk@mail.utexas.edu Abstract We represent natural language semantics by combining logical and distributional in- formation in probabilistic logic. We use Markov Logic Networks (MLN) for the RTE task, and Probabilistic Soft Logic (PSL) for the STS task. The system is evaluated on the SICK dataset. Our best system achieves 73% accuracy on the RTE task, and a Pearson’s correlation of 0.71 on the STS task. 1 Introduction Textual Entailment systems based on logical infer- ence excel in correct reasoning, but are often brit- tle due to their inability to handle soft logical in- ferences. Systems based on distributional seman- tics excel in lexical and soft reasoning, but are un- able to handle phenomena like negation and quan- tifiers. We present a system which takes the best of both approaches by combining distributional se- mantics with probabilistic logical inference. Our system builds on our prior work (Belt- agy et al., 2013; Beltagy et al., 2014a; Beltagy and Mooney, 2014; Beltagy et al., 2014b). We use Boxer (Bos, 2008), a wide-coverage semantic analysis tool to map natural sentences to logical form. Then, distributional information is encoded in the form of inference rules. We generate lexical and phrasal rules, and experiment with symmetric and asymmetric similarity measures. Finally, we use probabilistic logic frameworks to perform in- ference, Markov Logic Networks (MLN) for RTE, and Probabilistic Soft Logic (PSL) for STS. This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ 2 Background 2.1 Logical Semantics Logic-based representations of meaning have a long tradition (Montague, 1970; Kamp and Reyle, 1993). They handle many complex semantic phe- nomena such as relational propositions, logical operators, and quantifiers; however, they can not handle “graded” aspects of meaning in language because they are binary by nature. 2.2 Distributional Semantics Distributional models use statistics of word co- occurrences to predict semantic similarity of words and phrases (Turney and Pantel, 2010; Mitchell and Lapata, 2010), based on the obser- vation that semantically similar words occur in similar contexts. Words are represented as vec- tors in high dimensional spaces generated from their contexts. Also, it is possible to compute vec- tor representations for larger phrases composition- ally from their parts (Mitchell and Lapata, 2008; Mitchell and Lapata, 2010; Baroni and Zampar- elli, 2010). Distributional similarity is usually a mixture of semantic relations, but particular asym- metric similarity measures can, to a certain ex- tent, predict hypernymy and lexical entailment distributionally (Kotlerman et al., 2010; Lenci and Benotto, 2012; Roller et al., 2014). Distribu- tional models capture the graded nature of mean- ing, but do not adequately capture …

Content truncated to 500 words