SOFTCARDINALITY: Learning to Identify Directional Cross-Lingual Entailment from Cardinalities and SMT

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 34–38, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SOFTCARDINALITY: Learning to Identify Directional Cross-Lingual Entailment from Cardinalities and SMT Sergio Jimenez, Claudia Becerra Universidad Nacional de Colombia Ciudad Universitaria, edificio 453, oficina 114 Bogotá, Colombia sgjimenezv@unal.edu.co cjbecerrac@unal.edu.co Alexander Gelbukh CIC-IPN Av. Juan Dios Bátiz, Av. Mendizábal, Col. Nueva Industrial Vallejo CP 07738, DF, México gelbukh@gelbukh.com Abstract In this paper we describe our system submit- ted for evaluation in the CLTE-SemEval-2013 task, which achieved the best results in two of the four data sets, and finished third in av- erage. This system consists of a SVM clas- sifier with features extracted from texts (and their translations SMT) based on a cardinality function. Such function was the soft cardinal- ity. Furthermore, this system was simplified by providing a single model for the 4 pairs of languages obtaining better (unofficial) re- sults than separate models for each language pair. We also evaluated the use of additional circular-pivoting translations achieving results 6.14% above the best official results. 1 Introduction The Cross-Lingual Textual Entailment (CLTE) task consists in determining the type of directional en- tailment (i.e. forward, backward, bidirectional or no-entailment) between a pair of texts T1 and T2, each one written in different languages (Negri et al., 2013). The texts and reference annotations for this task were obtained through crowdsourcing applied to simpler sub-tasks (Negri et al., 2011). CLTE has as main applications content synchronization and aggregation in different languages (Mehdad et al., 2012; Duh et al., 2013). We participated in the first evaluation of this task in 2012 (Negri et al., 2012), achieving third place on average among 29 partici- pating systems (Jimenez et al., 2012). Since in the CLTE task text pairs are in different languages, in our system, all comparisons made be- tween two texts imply that one of them was written by a human and the other is a translation provided by statistical machine translation (SMT). Our approach is based on an SVM classifier (Cortes and Vapnik, 1995) whose features were cardinalities combined with similarity scores. That system was motivated by the fact that most text similarity functions are symmetric, e.g. Edit Distance (Levenshtein, 1966), longest common sub-sequence (Hirschberg, 1977), Jaro-Winkler similarity (Winkler, 1990), cosine sim- ilarity (Salton et al., 1975). Thus, the use of these functions as only resource seems counter-intuitive since CLTE task is asymmetric for the forward and backward entailment classes. Moreover, cardinality is the central component of the resemblance coefficients such as Jaccard, Dice, overlap, etc. For instance, if T1 and T2 are texts represented as bag of words, it is only necessary to know the cardinalities |T1|, |T2| and |T1 ∩ T2| to ob- tain a similarity score using a resemblance coeffi- cient such as the Dice’s coefficient (i.e. 2 · |T1 ∩ T2|/(|T1| + |T2|)). Therefore, the idea is to use the individual cardinalities to enrich a set of features ex- …

Content truncated to 500 words