HulTech: A General Purpose System for Cross-Level Semantic Similarity based on Anchor Web Counts

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 305–308, Dublin, Ireland, August 23-24, 2014. HulTech: A General Purpose System for Cross-Level Semantic Similarity based on Anchor Web Counts Jose G. Moreno Rumen Moraliyski Asma Berrezoug Ga¨el Dias Normandie University UNICAEN, GREYC CNRS F-14032 Caen, France firstname.lastname@unicaen.fr Abstract This paper describes the HULTECH team par- ticipation in Task 3 of SemEval-2014. Four different subtasks are provided to the partici- pants, who are asked to determine the semantic similarity of cross-level test pairs: paragraph- to-sentence, sentence-to-phrase, phrase-to- word and word-to-sense. Our system adopts a unified strategy (general purpose system) to calculate similarity across all subtasks based on word Web frequencies. For that purpose, we define ClueWeb InfoSimba, a cross-level similarity corpus-based metric. Results show that our strategy overcomes the proposed base- lines and achieves adequate to moderate re- sults when compared to other systems. 1 Introduction Similarity between text documents is considered a challenging task. Recently, many works concentrate on the study of semantic similarity for multi-level text doc- uments (Pilehvar et al., 2013), but skipping the cross- level similarity task. In the later, the underlying idea is that text similarity can be considered between pairs of text documents at different granularities levels: para- graph, sentence, phrase or word. One obvious partic- ularity of this task is that text pairs may not share the same characteristics of size, context or structure, i.e., the granularity level. In task 3 of SemEval-2014, two different strategies have been proposed to solve this issue. On the one hand, participants may propose a combination of indi- vidual systems, each one solving a particular subtask. On the other hand, a general purpose system may be proposed, which deals with all the subtasks following the exact same strategy. In this paper, we describe a language-independent corpus-based general purpose system, which relies on a huge freely available Web collection called Anchor- ClueWeb12 (Hiemstra and Hauff, 2010). In particular, we calculate ClueWeb InfoSimba1 a cross-level seman- This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ 1It is a Web version of InfoSimba (Dias et al., 2007). tic similarity based on word-word frequencies. Indeed, these frequencies are captured by the use of a colloca- tion metric called SCP2 (Silva et al., 1999), which has similar properties as the well studied PMI-IR (Turney, 2001) but does not over-evaluate rare events. Our system outputs a normalized (between 0 and 1) similarity value between two pieces of texts. However, the subtasks proposed in task 3 of SemEval-2014 in- clude a different scoring scale between 0 and 4. To solve this issue, we applied linear, polynomial and ex- ponential regressions as three different runs. Results show that our strategy overcomes the proposed base- lines and achieves adequate to moderate results when compared to other systems. 2 System Description Our system is based on a reduced version of the ClueWeb12 dataset called Anchor ClueWeb12 and …

Content truncated to 500 words