FBK: Cross-Lingual Textual Entailment Without Translation

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 701–705, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics FBK: Cross-Lingual Textual Entailment Without Translation Yashar Mehdad FBK-irst Trento , Italy mehdad@fbk.eu Matteo Negri FBK-irst Trento , Italy negri@fbk.eu Jos´e Guilherme C. de Souza FBK-irst & University of Trento Trento, Italy desouza@fbk.eu Abstract inThis paper overviews FBK’s participation the Cross-Lingual Textual Entailment for Content Synchronization task organized within SemEval-2012. Our participation is characterized by using cross-lingual matching features extracted from lexical and semantic phrase tables and dependency relations. The features are used for multi-class and binary classification using SVMs. Using a combi- nation of lexical, syntactic, and semantic fea- tures to create a cross-lingual textual entail- ment system, we report on experiments over the provided dataset. Our best run achieved an accuracy of 50.4% on the Spanish-English dataset (with the average score and the me- dian system respectively achieving 40.7% and 34.6%), demonstrating the effectiveness of a “pure” cross-lingual approach that avoids in- termediate translations. 1 Introduction So far, cross-lingual textual entailment (CLTE) (Mehdad et al., 2010) has been applied to: available TE datasets (“YES”/“NO” uni-directionali) relations between monolingual pairs) transformed into their cross-lingual counterpart by translating the hypotheses into other languages (Negri and Mehdad, 2010), and ii) machine translation evalu- ation datasets (Mehdad et al., 2012b). The content synchronization task represents a challenging appli- cation scenario to test the capabilities of CLTE sys- tems, by proposing a richer inventory of phenomena (i.e. “Bidirectional”/“Forward”/“Backward”/“No entailment” multi-directional entailment relations). Multi-directional CLTE recognition can be seen as the identification of semantic equivalence and in- formation disparity between two topically related sentences, at the cross-lingual level. This is a core aspect of the multilingual content synchronization task, which represents a challenging application sce- nario for a variety of NLP technologies, and a shared research framework for the integration of semantics and MT technology. The CLTE methods proposed so far adopt either a “pivoting approach” (translation of the two in- put texts into the same language, as in (Mehdad et al., 2010)), or an “integrated solution” that exploits bilingual phrase tables to capture lexical relations and contextual information (Mehdad et al., 2011). The promising results achieved with the integrated approach still rely on phrasal matching techniques that disregard relevant semantic aspects of the prob- lem. By filling this gap integrating linguistically motivated features, in our participation, we propose an approach that combines lexical, syntactic and se- mantic features within a machine learning frame- work (Mehdad et al., 2012a). Our submitted runs have been produced by train- ing and optimizing multiclass and binary SVM clas- sifiers, over the Spanish-English (Spa-Eng) devel- opment set. In both cases, our results were posi- tive, showing significant improvements over the me- dian systems and average scores obtained by partic- ipants. The overall results confirm the difficulty of the task, and the potential of our approach in com- bining linguistically motivated features in a “pure” cross-lingual approach that avoids the recourse to external MT components. 701 2 Experiments …

Content truncated to 500 words