On Machine Translation of User Reviews

Proceedings of Recent Advances in Natural Language Processing, pages 1109–1118 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_124 1109 On Machine Translation of User Reviews Maja Popovi´c, Andy Way ADAPT Centre School of Computing DCU, Ireland name.surname@adaptcentre.ie Alberto Poncelas Rakuten Asia Singapore aponcelas2@hotmail.com Marija Brki´c Bakari´c Department of Informatics University of Rijeka, Croatia mbrkic@uniri.hr Abstract This work investigates neural machine trans- lation (NMT) systems for translating English user reviews into Croatian and Serbian, two similar morphologically complex languages. Two types of reviews are used for testing the systems: IMDb movie reviews and Amazon product reviews. Two types of training data are explored: large out-of-domain bilingual parallel corpora, as well as small synthetic in-domain parallel cor- pus obtained by machine translation of mono- lingual English Amazon reviews into the tar- get languages. Both automatic scores and hu- man evaluation show that using the synthetic in-domain corpus together with a selected sub- set of out-of-domain data is the best option. Separated results on IMDb and Amazon re- views indicate that MT systems perform dif- ferently on different review types so that user reviews generally should not be considered as a homogeneous genre. Nevertheless, more de- tailed research on larger amount of different reviews covering different domains/topics is needed to fully understand these differences. 1 Introduction Machine translation (MT) has evolved very rapidly since the emergence of neural approaches in 2015, and it is being used for different genres and do- mains. Every year, evaluation campaigns which include both human and automatic evaluation are carried out with the goal of advancing the state of the art. The most well-known is the WMT shared task1 which focuses on news articles and (since 2016) on biomedical texts, and both can be con- sidered as instances of “formal written text". The IWSLT evaluation campaign2, on the other hand, 1http://www.statmt.org/wmt20/ 2http://workshop2019.iwslt.org/index. php focuses on the translation of TED talks, and some European projects (TraMOOC, transLectures) in- vestigated the translation of online lectures. In both cases, the text can be considered to be “formal speech", with the challenges of dealing with charac- teristics of spoken language and speech recognition output. Recently, interest in the translation of user- generated content in the form of “informal written text" has been increasing. For example, JSALT 2019 workshop3 focused on translation of very noisy text content originating from sources like WhatsApp, Twitter and Reddit. In this work, we focus on a different type of writ- ten user-generated content, namely user reviews. While the style is not as colloquial and noisy as that of Twitter or of other similar sources, it cer- tainly is much less formal than news texts or other sources that have been investigated traditionally in the MT community. There are also important appli- cations for focusing on this kind of data, both from commercial and from user perspective. More and more companies are expanding into multinational markets, and user reviews of products have become an important asset for online transactions and a feature that many customers expect to find. And in the era …

Content truncated to 500 words