Parallel Sentence Mining by Constrained Decoding

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1672–1678 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1672 Parallel Sentence Mining by Constrained Decoding Pinzhen Chen∗ Nikolay Bogoychev∗ Kenneth Heafield Faheem Kirefu School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB {pinzhen.chen, n.bogoych}@ed.ac.uk, {kheafiel, fkirefu}@inf.ed.ac.uk Abstract We present a novel method to extract parallel sentences from two monolingual corpora, us- ing neural machine translation. Our method relies on translating sentences in one cor- pus, but constraining the decoding by a pre- fix tree built on the other corpus. We argue that a neural machine translation system by it- self can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search. When bench- marked on the BUCC shared task, our method achieves results comparable to other submis- sions. 1 Introduction Having large and high-quality parallel corpora is critical for neural machine translation (NMT). One way to create such a resource is to mine the web (Resnik and Smith, 2003). Once texts are crawled from the web, they form large collections of data in different languages. To find parallel sentences, a natural way is to score sentence similarity between all possible sentence pairs and extract the top- scoring ones. This poses two major challenges: 1. Accurately determining the semantic similar- ity of a sentence pair in two languages. 2. Efficiently scoring sentence similarity for all possible pairs across two languages. Scoring each source sentence against each tar- get sentence results in unaffordable quadratic time complexity. A typical workflow reduces the search complexity in a coarse-to-fine manner by aligning documents then aligning sentences within docu- ments (Uszkoreit et al., 2010). However, trans- lated websites may not have matching document structures. More recent methods focus on direct sentence alignment. The results from Building and Using ∗Equal contribution. Comparable Corpora (BUCC) shared task show that direct sentence alignment can be done by sentence-level lexical comparison, neural compar- ison or a combination of the two (Zweigenbaum et al., 2017, 2018). A state-of-the-art method maps all sentences to multilingual sentence em- beddings and compares them using vector similar- ity (Artetxe and Schwenk, 2019). Such sentence embeddings are produced by neural encoders, but the rise of the attention mechanism demonstrates that sentence embeddings alone are insufficient to obtain full translation quality (Bahdanau et al., 2015). To exploit quality gains from the attention mechanism, we propose to use a full NMT sys- tem with attention to score potentially parallel sen- tences. The way we avoid pairwise scoring is in- spired by constrained decoding in NMT, where the choice of output tokens is constrained to a pre- defined list (Hokamp and Liu, 2017). Our method works as follows: We designate one language as source and one language as target, and build a trie over all target sentences. Then we translate each source sentence to the target language, but constrain left-to-right beam search to follow the trie. In other words, every translation hypothesis is a …

Content truncated to 500 words