Fast, Scalable Phrase-Based SMT Decoding

Fast, Scalable Phrase-Based SMT Decoding hieu@moses-mt.org s1031254@sms.ed.ac.uk lanes@illinois.edu junczys@amu.edu.pl Hieu�Hoang Moses�Machine�Translation�CIC,�UK Ni�olay�Bogoychev University�of�Edinburgh,�Scotland Lane�Schwartz University�of�Illinois,�USA Marcin�Junczys-Dowmunt Adam�Mickiewicz�University Abstract The utilization of statistical machine translation (SMT) has grown enormously over the last decade, many using open-source software developed by the NLP community. As commercial use has increased, there is need for software that is optimized for commercial requirements, in particular, fast phrase-based decoding and more efficient utilization of modern multicore servers. In this paper we re-examine the major components of phrase-based decoding and decoder im- plementation with particular emphasis on speed and scalability on multicore machines. The result is a drop-in replacement for the Moses decoder which is up to fifteen times faster and scales monotonically with the number of cores. 1 Introduction SMT has steadily progressed from a research discipline to commercial viability during the past decade as can be seen from services such as the Google and Microsoft Translation services. As well as general purpose services such as these, there is a large number of companies that offer customized translation systems, as well as companies and organization that implement in-house solutions. Many of these customized solutions use Moses as their SMT engine. For many users, decoding is the most time-critical part of the translation process. Making use of the multiple cores that are now ubiquitous in todays servers is a common strategy to ameliorate this issue. However, it has been noticed that the Moses decoder, amongst others, is unable to efficiently use multiple cores (Fern´andez et al., 2016). That is, decoding speed does not substantially increase when more cores are used, in fact, it may actually decrease when using more cores. There have been speculation on the causes of the inefficiency as well as potential remedies. This paper is the first we know of that focuses on improving decoding speed on multicore servers. We take a holistic approach to solving this issue, creating a decoder that is optimized for multi-core processing speed by concentrating on four main areas: 1. Faster memory management of data-structures through the use of customized memory pools 2. Exploring alternatives to cardinality-based hypothesis stack configuration ������������������������������������������������������ ���������������������������������������� 3. Re-examining the efficiency of phrase-table lookup using translation rule caching and data compression 4. Integrating the lexicalized re-ordering model into the phrase-table, thus eliminating the need for independent random lookup this model The result is a decoder that is significantly faster than the Moses baseline for single- threaded operation, and scales with the number of cores. We will maintain the Moses decoder’s embarrassingly parallel, one sentence-per-thread decoding framework. As far as possible, model scores and functionality are compatible with Moses to aid comparison and ease transition for existing users. The source code is available in the existing Moses repository1. The rest of the paper will be broken up into the following sections. The rest of this section will discuss prior work and an outline of the phrase-based model. Section 2 will then describe the modifications to improve decoding speed. We describe the experiment setup in Section 3 …

Content truncated to 500 words