Multilingual Corpus Creation for Multilingual Semantic Similarity Task

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4190–4196 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4190 Multilingual Corpus Creation for Multilingual Semantic Similarity Task Mahtab Ahmed1, Chahna Dixit2, Robert E. Mercer1, Atif Khan2, Muhammad Rifayat Samee1, Felipe Urra1 1University of Western Ontario, 2Messagepoint Inc. mahme255@uwo.ca, chahna.dixit@messagepoint.ai, mercer@csd.uwo.ca, atif.khan@messagepoint.ai, msamee@uwo.ca, furra@uwo.ca Abstract In natural language processing, the performance of a semantic similarity task relies heavily on the availability of a large corpus. Various monolingual corpora are available (mainly English); but multilingual resources are very limited. In this work, we describe a semi- automated framework to create a multilingual corpus which can be used for the multilingual semantic similarity task. The similar sentence pairs are obtained by crawling bilingual websites, whereas the dissimilar sentence pairs are selected by applying topic modeling and an Open-AI GPT model on the similar sentence pairs. We focus on websites in the government, insurance, and banking domains to collect English-French and English-Spanish sentence pairs; however, this corpus creation approach can be applied to any other industry vertical provided that a bilingual website exists. We also show experimental results for multilingual semantic similarity to verify the quality of the corpus and demonstrate its usage. Keywords: Multilingual Corpus, Multilingual Semantic Similarity, Web Crawling 1. Introduction Semantic similarity, one of the important natural language processing (NLP) tasks, aims to measure the distance be- tween two given content pieces in terms of their meaning. Traditionally, WordNet-based similarity measures such as Lin, Resnik, Jiang and Conrath (Budanitsky and Hirst, 2006) as well as statistical approaches including Latent Se- mantic Analysis (LSA) (Landauer et al., 2013) and Point- wise Mutual Information (PMI) (Zhao et al., 2014) have been used to solve this problem. Recently with the ad- vent of deep learning, the use of deep neural networks has gained popularity in solving this task; for example Siamese recurrent networks (Mueller and Thyagarajan, 2016) and convolutional neural networks (Shao, 2017). However, a major factor affecting the success of deep networks is the availability of substantially large and good quality corpora (Kiros et al., 2015; Devlin et al., 2019). The most popular benchmark dataset for semantic sim- ilarity is the Semantic Textual Similarity (STS) dataset from SemEval tasks. The latest STS17 dataset (Cer et al., 2017) includes monolingual as well as cross-lingual sentence pairs for English, Arabic and Spanish languages. Nonetheless, the STS corpus requires a classification score ranging from 0 to 5 measuring the degree of similarity be- tween the sentence pairs. We approach multilingual seman- tic similarity as a binary classification problem which has required us to collect a large corpus of our own based on the domain and language requirements of our application. The collection of an entirely new and large corpus in it- self is a challenging task; more specifically, textual data for NLP problems require human expertise and domain knowl- edge of the application. Above all, the acquisition of a mul- tilingual corpus also demands some amount of …

Content truncated to 500 words