CDA: a Cost Efficient Content-based Multilingual Web Document Aligner

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3053–3061 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3053 CDA: a Cost Efficient Content-based Multilingual Web Document Aligner Thuy Vu Amazon Alexa AI Manhattan Beach, CA, USA thuyvu@amazon.com Alessandro Moschitti Amazon Alexa AI Manhattan Beach, CA, USA amosch@amazon.com Abstract We introduce a Content-based Document Alignment approach (CDA), an efficient method to align multilingual web documents based on content in creating parallel training data for machine translation (MT) systems op- erating at the industrial level. CDA works in two steps: (i) projecting documents of a web domain to a shared multilingual space; then (ii) aligning them based on the similarity of their representations in such space. We lever- age lexical translation models to build vector representations using TF×IDF. CDA achieves performance comparable with state-of-the-art systems in the WMT-16 Bilingual Document Alignment Shared Task benchmark while op- erating in multilingual space. Besides, we cre- ated two web-scale datasets to examine the ro- bustness of CDA in an industrial setting involv- ing up to 28 languages and millions of docu- ments. The experiments show that CDA is ro- bust, cost-effective, and is significantly supe- rior in (i) processing large and noisy web data and (ii) scaling to new and low-resourced lan- guages. 1 Introduction Online machine translation (MT) services require industrial-scale training data, i.e., significantly large and high-quality parallel sentences, to build accurate models. Exploiting the web for multilin- gual content has become a usual strategy in collect- ing large-scale parallel sentences for MT (Uszkor- eit et al., 2010; Smith et al., 2013; Buck and Koehn, 2016a). Structural Translation Recognition for Acquiring Natural Data (STRAND) (Resnik and Smith, 2003) is a standard pipeline to extract paral- lel data from the web, consisting in three steps: (i) bilingual document alignment for an input set of documents, (ii) sentence alignment for each aligned document pair, and (iii) sentence filtering for non- translation or boilerplate cleaning. The first step of identifying bilingual documents is technically challenging and made more complicated by the presence of large and noisy documents from web data. In the WMT-16 Bilingual Document Alignment Shared Task (WMT16-BDAST), two standard ap- proaches for identifying parallel pages1 were stud- ied: 1) URL matching heuristic (Smith et al., 2013) as a baseline and 2) content similarity as a solu- tion to maximize the performance in identifying parallel documents. The benchmark on English- French document alignment task shows that the best top-1 recall (R@1) for each approach are 59.8% and 89.1%, respectively, as evaluated on the test set (Buck and Koehn, 2016a,b). The results, albeit conducted within English-French setting, in- dicate that leveraging document content can lead to a significant increase, up to 30 percent points in recall, and contributes ∼50% novel bilingual document pairs. The URL matching heuristic approach, named URL, identifies parallel pages using language iden- tifiers, typically from ISO 639, annotated in the addresses. Pages, or web-documents, in differ- ent languages from a domain are …

Content truncated to 500 words