CMU’s Machine Translation System for IWSLT 2019

CMU’s Machine Translation System for IWSLT 2019 Tejas Srinivasan, Ramon Sanabria, Florian Metze Language Technologies Institute Carnegie Mellon University, USA tsriniva@cs.cmu.edu Abstract In Neural Machine Translation (NMT) the usage of sub- words and characters as source and target units offers a sim- ple and flexible solution for translation of rare and unseen words. However, selecting the optimal subword segmenta- tion involves a trade-off between expressiveness and flexibil- ity, and is language and dataset-dependent. We present Block Multitask Learning (BMTL), a novel NMT architecture that predicts multiple targets of different granularities simulta- neously, removing the need to search for the optimal seg- mentation strategy. Our multi-task model exhibits improve- ments of up to 1.7 BLEU points on each decoder over single- task baseline models with the same number of parameters on datasets from two language pairs of IWSLT15 and one from IWSLT19. The multiple hypotheses generated at different granularities can also be combined as a post-processing step to give better translations. 1. Introduction Neural Machine Translation (NMT) [1, 2, 3] provides a sim- ple, end-to-end framework for translating text from one lan- guage to another. NMT approaches have largely outper- formed and replaced previous statistical translation methods. Traditionally, NMT systems used words as source and tar- get units, which have three main disadvantages. First, word- based models are unable to translate rare and out of vocab- ulary (OOV) words in the source language. Second, they can not produce unseen target words, such as morphologi- cal variants of observed words (e.g., deriving realistic from real). Third, they have to handle large source and target language vocabularies (i.e., large look-up matrices), which makes them less scalable in term of computation and mem- ory. A large vocabulary also implies data sparsity where the number of tokens is not balanced. A common solution for the problems mentioned above is to perform word segmentation. The Byte-Pair Encod- ing (BPE) algorithm [4] groups units together according to their frequency. By presetting the desired vocabulary size, the BPE algorithm generates a segmentation of the data by representing words as a collection of subword units. More recently, [5] proposed a forward probabilistic subword seg- mentation algorithm which is based on a unigram language model, in contrast to the deterministic BPE algorithm. Un- �I �flew �on �Air �Force �Tw o �for �eight �years �. Subword1k Subword300 Shared Encoder Subword10k PROJ BiLSTM … BiLSTM n <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="hP+6LrUf2d3tZaldqaQvEKMXyw=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+ gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odBu3wMYA6nMFXEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit> <latexit sha1_base64="NtnmDLJ1NgRz18QxoCqIYjLd6rc=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5 +6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPOMxspUP9I0wzDhIy1jKTg5q6kH1Zpf9xdimxCsoAYrNQbVr/4wFXmCmoTi1vYCP6Ow4IakUDgr93OLGRcTPsKeQ80TtGxGHTGLp0zZHFq3NHEFu7vFwVPrJ0mkbuZcBrb9Wxu/pf1copvw0LqLCfUYvlRnCtGKZtvzYbSoCA1dcCFkW5WJsbcEGum7IrIVhfeRPa1/XArwdNH0pwDhdwBQHcwB08QANaIADhBd7g3XvyXr2PZV1b3q3M/gj7/MHwtmLnw=</lat exit> <latexit sha1_base64="NtnmDLJ1NgRz18QxoCqIYjLd6rc=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5 +6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPOMxspUP9I0wzDhIy1jKTg5q6kH1Zpf9xdimxCsoAYrNQbVr/4wFXmCmoTi1vYCP6Ow4IakUDgr93OLGRcTPsKeQ80TtGxGHTGLp0zZHFq3NHEFu7vFwVPrJ0mkbuZcBrb9Wxu/pf1copvw0LqLCfUYvlRnCtGKZtvzYbSoCA1dcCFkW5WJsbcEGum7IrIVhfeRPa1/XArwdNH0pwDhdwBQHcwB08QANaIADhBd7g3XvyXr2PZV1b3q3M/gj7/MHwtmLnw=</lat exit> <latexit sha1_base64="ck1/v/JpOJELH0iIdgPbhaWErXY=">AB6HicbVBNT8JAEJ3iF+IX6tHLRmLibRe9Ej04hESCyTQkO0yhZXtndmpCGX+DFg8Z49Sd589+4QA8KvmSl/dmMjMvTAXxnW/ndLG5tb 2Tnm3srd/cHhUPT5p6yRTDH2WiER1Q6pRcIm+4UZgN1VI41BgJ5zczf3OEyrNE/lgpikGMR1JHnFGjZVaclCtuXV3AbJOvILUoEBzUP3qDxOWxSgNE1TrnuemJsipMpwJnFX6mcaUsgkdYc9SWPUQb4dEYurDIkUaJsSUMW6u+JnMZaT+PQdsbUjPWqNxf/83qZiW6CnMs0MyjZclGUCWISMv+aDLlCZsTUEsoUt7cSNqaKMmOzqdgQvNWX10n7qu65da/l1hq3RxlOINzuAQPrqEB9AEHxgPMrvDmPzovz7nwsW0tOMXMKf+B8/gDV0 Yzu</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> 1 <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="hP+6LrUf2d3tZaldqaQvEKMXyw=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+ gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odBu3wMYA6nMFXEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit> <latexit sha1_base64="2yIKfhykAKv0MhUqDiS4SywkL1Y=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPO MxspUP9I0wzDhIy1jKTg5qxkMqjW/7i/ENiFYQ1WagyqX/1hKvIENQnFre0FfkZhwQ1JoXBW7ucWMy4mfIQ9h5onaMNiMeiMXTpnyOLUuKOJLdzfLwqeWDtNIncz4TS269nc/C/r5RTfhoXUWU6oxfKjOFeMUjbfmg2lQUFq6oALI92sTIy54YJcN2VXQrC+8ia0r+uBXw+aPpTgHC7gCgK4gTt4gAa0QADC7zBu/fkvXofy7q2vFVvZ/BH3ucPaQSLYg=</latexit> <latexit sha1_base64="2yIKfhykAKv0MhUqDiS4SywkL1Y=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPO MxspUP9I0wzDhIy1jKTg5qxkMqjW/7i/ENiFYQ1WagyqX/1hKvIENQnFre0FfkZhwQ1JoXBW7ucWMy4mfIQ9h5onaMNiMeiMXTpnyOLUuKOJLdzfLwqeWDtNIncz4TS269nc/C/r5RTfhoXUWU6oxfKjOFeMUjbfmg2lQUFq6oALI92sTIy54YJcN2VXQrC+8ia0r+uBXw+aPpTgHC7gCgK4gTt4gAa0QADC7zBu/fkvXofy7q2vFVvZ/BH3ucPaQSLYg=</latexit> <latexit sha1_base64="1enTr8x+dPZYKeA2su7IqvnUdLk=">AB6HicbVBNT8JAEJ3iF+IX6tHLRmLibRe9Ej04hESCyTQkO0yhZXtndmpCGX+DFg8Z49Sd589+4QA8KvmSl/dmMjMvTAXxnW/ndLG5tb2Tnm3srd/cHhUPT5p6yRTDH2WiER1Q6pRcIm+4UZgN1VI41BgJ5zczf3 OEyrNE/lgpikGMR1JHnFGjZVa3qBac+vuAmSdeAWpQYHmoPrVHyYsi1EaJqjWPc9NTZBTZTgTOKv0M40pZRM6wp6lksaog3x6IxcWGVIokTZkoYs1N8TOY21nsah7YypGetVby7+5/UyE90EOZdpZlCy5aIoE8QkZP41GXKFzIipJZQpbm8lbEwVZcZmU7EheKsvr5P2Vd1z617LrTVuizjKcAbncAkeXEMD7qEJPjBAeIZXeHMenRfn3flYtpacYuYU/sD5/AF5XYyx</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> UniLSTM decoder PROJ Subword10k PROJ Unit-specific CE Loss Unit-specific CE Loss Unit-specific CE Loss …

Content truncated to 500 words