Why don’t people use character-level machine translation?

Findings of the Association for Computational Linguistics: ACL 2022, pages 2470 - 2485 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Why don’t people use character-level machine translation? Jindˇrich Libovický1 and Helmut Schmid2 and Alexander Fraser2 1 Faculty of Mathematics and Physics, Charles Univeristy, Prague, Czech Republic 2 Center for Information and Speech Processing, LMU Munich, Germany libovicky@ufal.mff.cuni.cz {schmid, fraser}@cis.lmu.de Abstract We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine trans- lation (MT). Despite evidence in the litera- ture that character-level systems are compa- rable with subword systems, they are virtu- ally never used in competitive setups in WMT competitions. We empirically show that even with recent modeling innovations in character- level natural language processing, character- level MT systems still struggle to match their subword-based counterparts. Character-level MT systems show neither better domain ro- bustness, nor better morphological generaliza- tion, despite being often so motivated. How- ever, we are able to show robustness towards source side noise and that translation quality does not degrade with increasing beam size at decoding time. 1 Introduction The progress in natural language processing (NLP) brought by deep learning is often narrated as remov- ing assumptions about the input data and letting the models learn everything end-to-end. One of the as- sumptions about input data that seems to resist this trend is (at least partially) linguistically motivated segmentation of input data in machine translation (MT) and NLP in general. For NMT, several papers have claimed parity of character-based methods with subword models, highlighting advantageous features of such systems. Very recent examples include Gao et al. (2020); Ba- nar et al. (2020); Li et al. (2021). Despite this, character-level methods are rarely used as strong baselines in research papers and shared task sub- missions, suggesting that character-level models might have drawbacks that are not sufficiently ad- dressed in the literature. In this paper, we examine what the state of the art in character-level MT really is. We survey ex- isting methods and conduct a meta-analysis of the input segmentation methods used in WMT shared task submissions. We then systematically compare the most recent character-processing architectures, some of them taken from general NLP research and used for the first time in MT. Further, we propose an alternative two-step decoder architec- ture that unlike standard decoders does not suffer from a slow-down due to the length of character sequences. Following the recent findings on MT decoding, we evaluate different decoding strategies in the character-level context. Many previous studies on character-level MT drew their conclusions from experiments on rather small datasets and focused only on quantitatively assessed translation quality without further analy- sis. To compensate for this, we revisit and system- atically evaluate the state-of-the-art approaches to character-level neural MT and identify their major strengths and weaknesses on large datasets. 2 Character-Level Neural MT Character-level processing was hardly possible within the statistical MT paradigm that assumed the existence of phrases consisting of semantically rich tokens that roughly correspond …

Content truncated to 500 words