Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1224–1234 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1224 Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation Viktor Hangya and Alexander Fraser Center for Information and Language Processing LMU Munich, Germany {hangyav, fraser}@cis.lmu.de Abstract Mining parallel sentences from comparable corpora is important. Most previous work re- lies on supervised systems, which are trained on parallel data, thus their applicability is problematic in low-resource scenarios. Re- cent developments in building unsupervised bilingual word embeddings made it possible to mine parallel sentences based on cosine simi- larities of source and target language words. We show that relying only on this informa- tion is not enough, since sentences often have similar words but different meanings. We de- tect continuous parallel segments in sentence pair candidates and rely on them when min- ing parallel sentences. We show better mining accuracy on three language pairs in a standard shared task on artificial data. We also provide the first experiments showing that parallel sen- tences mined from real life sources improve unsupervised MT. Our code is available, we hope it will be used to support low-resource MT research. 1 Introduction The performance of machine translation has im- proved significantly recently, with some claims of even being close to human parity (Hassan et al., 2018), but a large amount of parallel data is re- quired for high quality systems. For many lan- guage pairs the size of the available training data is not adequate. Recently, developments in the field of unsupervised bilingual word embeddings (BWEs) made it possible to build MT systems without any parallel data. Both statistical (Lam- ple et al., 2018b; Artetxe et al., 2018b) and neu- ral (Artetxe et al., 2018c; Lample et al., 2018a) MT approaches were proposed which are promis- ing directions to overcome the data sparsity prob- lem. However, various issues of the approaches still have to be solved, e.g., better word reorder- ing during translation or tuning system parame- ters. For many interesting low resource language pairs, we do not have enough parallel data, but we do have access to sources of comparable mono- lingual text. In this paper we propose a strong unsupervised system for parallel sentence mining and show that the mined data improves the perfor- mance of unsupervised MT systems. Previously many approaches tackled the prob- lem of parallel sentence extraction but they were relying on different levels of bilingual signals ei- ther to build dictionaries (Grover and Mitra, 2017), parallel sentence classifiers (Bouamor and Saj- jad, 2018) or bilingual sentence representations (Schwenk, 2018). An unsupervised system was also proposed which only relied on unsupervised BWEs, thus no additional resources are needed (Hangya et al., 2018). We use this approach as our baseline and show that relying only on word sim- ilarity information leads to false positive sentence pairs, such as in this example: • The US dollar has a considerable role in the international …

Content truncated to 500 words