haLF: Comparing a Pure CDSM Approach with a Standard Machine Learning System for RTE

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 300–304, Dublin, Ireland, August 23-24, 2014. haLF: Comparing a Pure CDSM Approach with a Standard Machine Learning System for RTE Lorenzo Ferrone University of Rome “Tor Vergata” Via del Politecnico 1 00133 Roma, Italy lorenzo.ferrone@gmail.com Fabio Massimo Zanzotto University of Rome “Tor Vergata” Via del Politecnico 1 00133 Roma, Italy fabio.massimo.zanzotto@uniroma2.it Abstract In this paper, we describe our sub- mission to the Shared Task #1. We tried to follow the underlying idea of the task, that is, evaluating the gap of full-fledged recognizing textual en- tailment systems with respect to com- positional distributional semantic mod- els (CDSMs) applied to this task. We thus submitted two runs: 1) a sys- tem obtained with a machine learning approach based on the feature spaces of rules with variables and 2) a sys- tem completely based on a CDSM that mixes structural and syntactic infor- mation by using distributed tree ker- nels. Our analysis shows that, under the same conditions, the fully CDSM system is still far from being competi- tive with more complex methods. 1 Introduction Recognizing Textual Entailment is a largely explored problem (Dagan et al., 2013). Past challenges (Dagan et al., 2006; Bar-Haim et al., 2006; Giampiccolo et al., 2007) explored methods and models applied in complex and natural texts. In this context, machine learn- ing solutions show interesting results. The Shared Task #1 of SemEval instead wants to explore systems in a more controlled textual environment where the phenomena to model are clearer. The aim of the Shared Task is to study how RTE systems built upon composi- tional distributional semantic models behave This work is licenced under a Creative Commons At- tribution 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http://creativecommons.org/licenses/by/ 4.0/ with respect to the above tradition. We tried to capture this underlying idea of the task. In this paper, we describe our submission to the Shared Task #1. We tried to fol- low the underlying idea of the task, that is, evaluating the gap of full-fledged recognizing textual entailment systems with respect to compositional distributional semantic models (CDSMs) applied to this task. We thus sub- mitted two runs: 1) a system obtained with a machine learning approach based on the fea- ture spaces of rules with variables (Zanzotto et al., 2009) and 2) a system completely based on a CDSM that mixes structural and syntac- tic information by using distributed tree ker- nels (Zanzotto and Dell’Arciprete, 2012). Our analysis shows that, under the same condi- tions, the fully CDSM system is still far from being competitive with more complete meth- ods. The rest of the paper is organized as follows. Section 2 describes the full-fledged recognizing textual entailment system that is used for com- parison. Section 3 introduces a novel composi- tional distributional semantic model, namely, the distributed smoothed tree kernels, and the way this model is applied to the task of RTE. Section 4 describes the results …

Content truncated to 500 words