Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation

Proceedings of the 27th International Conference on Computational Linguistics, pages 1442–1453 Santa Fe, New Mexico, USA, August 20-26, 2018. 1442 Extracting Parallel Sentences with Bidirectional Recurrent Neural Networks to Improve Machine Translation Francis Gr´egoire Universit´e de Montr´eal francis.gregoire@rd.mila.quebec Philippe Langlais Universit´e de Montr´eal felipe@iro.umontreal.ca Abstract Parallel sentence extraction is a task addressing the data sparsity problem found in multilin- gual natural language processing applications. We propose a bidirectional recurrent neural net- work based approach to extract parallel sentences from collections of multilingual texts. Our experiments with noisy parallel corpora show that we can achieve promising results against a competitive baseline by removing the need of specific feature engineering or additional exter- nal resources. To justify the utility of our approach, we extract sentence pairs from Wikipedia articles to train machine translation systems and show significant improvements in translation performance. 1 Introduction Parallel corpora are a prerequisite for many multilingual natural language processing (NLP) applica- tions. Unfortunately, parallel data is available for a relatively small number of language pairs and for few specific domains. With the increasing amount of comparable corpora on the World Wide Web, a potential solution to alleviate the parallel data sparsity issue is to extract parallel sentences from this more abundant source of information. Therefore, the objective of parallel sentence extraction is to ex- tract parallel sentences from such comparable corpora to increase the quantity of parallel data and the range of the covered domains. Comparable corpora can be defined as collections of topic-aligned but non-sentence-aligned multilingual texts. Recent advances in deep learning architectures with recurrent neural networks (RNNs) have shown that they can successfully learn complex mappings from variable-length sequences to continuous vec- tor representations. While numerous natural language processing tasks have successfully applied those models, ranging from handwriting generation (Graves, 2013) to machine comprehension (Hermann et al., 2015), most of the multilingual efforts have been devoted to machine translation (Sutskever et al., 2014; Cho et al., 2014), although more research interests have been recently devoted to multilingual semantic textual similarity.1 In this paper, we propose a parallel sentence extraction system to measure the translational equivalence between sentences in two languages. Our system is based on bidirectional recurrent neural networks that can learn sentence representations in a shared vector space by explicitly maximizing the similarity be- tween parallel sentences. In contrast to previous approaches, by leveraging these continuous vector representation of sentences we remove the need to rely on multiple models and specific feature engineer- ing. Experiments on noisy parallel corpora show that our approach outperforms a competitive baseline. To justify the utility of our approach, we add the sentence pairs extracted from Wikipedia articles to a parallel corpus to train machine translation systems and show improvements in translation performance. Our experimental results lead us to believe that our system is a promising tool to create new aligned multilingual resources. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ 1http://alt.qcri.org/semeval2017/task2/ 1443 2 Related work Traditional systems developed to extract …

Content truncated to 500 words