UNIBA-CORE: Combining Strategies for Semantic Textual Similarity

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 169–175, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics UNIBA-CORE: Combining Strategies for Semantic Textual Similarity Annalina Caputo Pierpaolo Basile Department of Computer Science University of Bari Aldo Moro Via E. Orabona, 4 - 70125 Bari, Italy {annalina.caputo, pierpaolo.basile, giovanni.semeraro}@uniba.it Giovanni Semeraro Abstract This paper describes the UNIBA participation in the Semantic Textual Similarity (STS) core task 2013. We exploited three different sys- tems for computing the similarity between two texts. A system is used as baseline, which rep- resents the best model emerged from our pre- vious participation in STS 2012. Such system is based on a distributional model of seman- tics capable of taking into account also syn- tactic structures that glue words together. In addition, we investigated the use of two dif- ferent learning strategies exploiting both syn- tactic and semantic features. The former uses ensemble learning in order to combine the best machine learning techniques trained on 2012 training and test sets. The latter tries to overcome the limit of working with different datasets with varying characteristics by select- ing only the more suitable dataset for the train- ing purpose. 1 Introduction Semantic Textual Similarity is the task of comput- ing the similarity between any two given texts. The task, in its core formulation, aims at capturing the different kinds of similarity that emerge from texts. Machine translation, paraphrasing, synonym substi- tution or text entailment are some fruitful methods exploited for this purpose. These techniques, along with other methods for estimating the text similar- ity, were successfully employed via machine learn- ing approaches during the 2012 task. However, the STS 2013 core task (Agirre et al., 2013) differs from the 2012 formulation in that it provides a test set which is similar to the training, but not drawn from the same set of data. Hence, in order to generalize the machine learning models trained on a group of datasets, we investigate the use of combination strategies. The objective of combi- nation strategies, known under the name of ensem- ble learning, is that of reducing the bias-variance decomposition through reducing the variance error. Hence, this class of methods should be more ro- bust with respect to previously unseen data. Among the several ensemble learning alternatives, we ex- ploit the stacked generalization (STACKING) algo- rithm (Wolpert, 1992). Moreover, we investigate the use of a two-steps learning algorithm (2STEPSML). In this method the learning algorithm is trained us- ing only the dataset most similar to the instance to be predicted. The first step aims at predicting the dataset more similar to the given pair of texts. Then the second step makes use of the previously trained algorithm to predict the similarity value. The base- line for the evaluation is represented by our best sys- tem (DSM PERM) resulting from our participation in the 2012 task. After introducing the general mod- els behind our systems in Section …

Content truncated to 500 words