A Multilingual, Multi-style and Multi-granularity Dataset for Cross-language Textual Similarity Detection

A Multilingual, Multi-Style and Multi-Granularity Dataset for Cross-Language Textual Similarity Detection J´er´emy Ferrero1,2, Fr´ed´eric Agn`es1, Laurent Besacier2, Didier Schwab2 1 Compilatio, 276 rue du Mont Blanc, 74540 Saint-F´elix, France 2 LIG-GETALP, Univ. Grenoble Alpes, France {jeremy, frederic}@compilatio.net, {jeremy.ferrero, laurent.besacier, didier.schwab}@imag.fr Abstract In this paper we describe our effort to create a dataset for the evaluation of cross-language textual similarity detection. We present pre- existing corpora and their limits and we explain the various gathered resources to overcome these limits and build our enriched dataset. The proposed dataset is multilingual, includes cross-language alignment for different granularities (from chunk to document), is based on both parallel and comparable corpora and contains human and machine translated texts. Moreover, it includes texts written by multiple types of authors (from average to professionals). With the obtained dataset, we conduct a systematic and rigorous evaluation of several state-of-the-art cross-language textual similarity detection methods. The evaluation results are reviewed and discussed. Finally, dataset and scripts are made publicly available on GitHub: http://github.com/FerreroJeremy/Cross-Language-Dataset. Keywords: Dataset, Cross-language dataset, Evaluation, Cross-language similarity detection, Cross-language plagiarism detec- tion 1. Introduction Guibert and Michaut (2011) state that 34.5% of European students have already copied all or part of a document to present it as their own work. This confirms the work of the Josephson Institute (2011) and McCabe (2010) who esti- mate that more than 30% of American and Canadian stu- dents have already re-used Web sentences without citing their source; this is considered as plagiarism. “Plagiarism is an act of fraud to steal and pass off (the ideas or words of another) as one’s own without crediting the source to present as new and original an idea or product derived from an existing source” (Plagiarism.org, 2014). In addition, Internet expansion facilitates access to docu- ments in foreign languages and to increasingly efficient ma- chine translation tools. Consequently, a new kind of pla- giarism is becoming frequent: the Cross-Language Plagia- rism. It involves plagiarism by translation, i.e. a text has been plagiarized while being translated (manually or auto- matically). The challenge in detecting this kind of plagia- rism is that the suspicious document is in a language differ- ent from its source. In this relatively new field of research, no complete evalu- ation framework has been carried out and no sufficiently diversified reference dataset has been made available to enable more systematic and rigorous evaluations. Contributions. This paper presents our methodology to collect and build a reference dataset for the evaluation of cross-language textual similarity detection (made available to the research community). More precisely, the characteristics of our dataset are the following: • it is multilingual: French, English and Spanish; • it proposes cross-language alignment information at different granularities: document-level, sentence-level and chunk-level; • it is based on both parallel and comparable corpora; • it contains both human and machine translated text; • part of it has been altered (to make the cross-language similarity detection more complicated) while the rest remains without noise; • documents were written by multiple types of authors: …

Content truncated to 500 words