ReSiPC: a Tool for Complex Searches in Parallel Corpora

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7033–7037 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7033 ReSiPC: a tool for complex searches in parallel corpora Antoni Oliver, Bojana Mikeleni´c Universitat Oberta de Cataluya, University of Zagreb Barcelona (Catalonia-Spain), Zagreb (Croatia) aoliverg@uoc.edu, bmikelen@ffzg.hr Abstract In this paper, a tool specifically designed to allow for complex searches in large parallel corpora is presented. The formalism for the queries is very powerful as it uses standard regular expressions that allow for complex queries combining word forms, lemmata and POS-tags. As queries are performed over POS-tags, at least one of the languages in the parallel corpus should be POS-tagged. Searches can be performed in one of the languages or in both languages at the same time. The program is able to POS-tag the corpora using the Freeling analyzer through its Python API. ReSiPC is developed in Python version 3 and it is distributed under a free license (GNU GPL). The tool can be used to provide data for contrastive linguistics research and an example of use in a Spanish-Croatian parallel corpus is presented. ReSiPC is designed for queries in POS-tagged corpora, but it can be easily adapted for querying corpora containing other kinds of information. Keywords: parallel corpora, regular expressions, contrastive linguistics 1. Introduction Bilingual or multilingual corpora can be parallel (also known as translation corpora) or comparable, where the former contain texts in one language and their translations into one or more others and the latter contain original texts in two or more languages. For this paper, we will be fo- cusing only on parallel corpora. Johannson (2007, 301) lists some of the areas where multilingual corpora are used: contrastive linguistics, language typology, translation stud- ies, including translator education, bilingual lexicography, foreign-language teaching and natural language processing, including machine translation. For the latter, parallel cor- pora are often used to train machine translation models, statistical (Koehn et al., 2007) or neural (Bahdanau et al., 2015) and also for the purposes of bilingual terminology extraction (Lefever et al., 2009). Here, we are especially interested in their application in contrastive linguistics. Contrastive linguistics or contrastive analysis is a compar- ison of two or more languages in order to show structural similarities and differences between them. The contrastive approach has a long tradition in Croatian linguistics, dating from the works of Rudolf Filipovi´c, most importantly his “Yugoslav Serbo-Croatian – English Contrastive Project” started in 1968 at the Institute of Linguistics of the Univer- sity of Zagreb. For the purposes of this project and oth- ers to follow, one of the first parallel corpora ever was cre- ated, the so-called Zagreb Version of the Brown Corpus. In Filipovi´c (1969), it is explained how the Brown Corpus was shortened, translated into Serbo-Croatian and gram- matically tagged. This was done because the importance of translation for isolation of structures to be contrasted was observed from the very beginning (Filipovi´c, 1985, 23-4). Considering the use of corpora in …

Content truncated to 500 words