BUAP: N-gram based Feature Evaluation for the Cross-Lingual Textual Entailment Task

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 124–127, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics BUAP: N-gram based Feature Evaluation for the Cross-Lingual Textual Entailment Task Darnes Vilari˜no, David Pinto, Sa´ul Le´on, Yuridiana Alem´an, Helena G´omez-Adorno Benem´erita Universidad Aut´onoma de Puebla Faculty of Computer Science 14 Sur y Av. San Claudio, CU Puebla, Puebla, M´exico {darnes, dpinto, saul.leon, candy.aleman, helena.gomez}@cs.buap.mx Abstract This paper describes the evaluation of differ- ent kinds of textual features for the Cross- Lingual Textual Entailment Task of SemEval 2013. We have counted the number of N- grams for three types of textual entities (char- acter, word and PoS tags) that exist in the pair of sentences from which we are inter- ested in determining the judgment of textual entailment. Difference, intersection and dis- tance (Euclidian, Manhattan and Jaccard) of N-grams were considered for constructing a feature vector which is further introduced in a support vector machine classifier which al- lows to construct a classification model. Five different runs were submitted, one of them considering voting system of the previous four approaches. The results obtained show a per- formance below the median of six teams that have participated in the competition. 1 Introduction The cross-lingual textual entailment (CLTE), re- cently proposed by (Mehdad et al., 2012) and (Mehdad et al., 2011), is an extension of the tex- tual entailment task (Dagan and Glickman, 2004). Formally speaking, given a pair of topically related text fragments (T1 and T2 which are assumed to be TRUE statements) written in different languages, the CLTE task consists of automatically annotating it with one of the following entailment judgments: • bidirectional (T1 → T2 & T1 ← T2): the two fragments entail each other (semantic equiva- lence); • forward (T1 → T2 & T1 ↚ T2): unidirec- tional entailment from T1 to T2; • backward (T1 ↛ T2 & T1 ← T2): unidirec- tional entailment from T2 to T1; • no entailment (T1 ↛ T2 & T1 ↚ T2): there is no entailment between T1 and T2 in both directions; The Cross-lingual datasets evaluated were avail- able for the following language combinations (T1- T2): • Spanish-English (SPA-ENG) • German-English (DEU-ENG) • Italian-English (ITA-ENG) • French-English (FRA-ENG) In this paper we describe the evaluation of differ- ent features extracted from each pair of topically re- lated sentences. N-grams of characters, words and PoS tags were counted with the aim of constructing a representative vector for each judgment entailment (FORWARD, BACKWARD, BI-DIRECTIONAL or NO-ENTAILMENT). The resulting vectors were fed into a supervised classifier based on Support Vector Machines (SVM)1 which attempted to con- struct a classification model. The description of the features and the vectorial representation is given in Section 2. The obtained results are shown and di- cussed in Section 3. Finally, the findings of this work are given in Section 4. 1We have employed the implementation of the Weka tool (Hall et al., 2009). 124 2 Experimental …

Content truncated to 500 words