Multitask Learning For Different Subword Segmentations In Neural Machine Translation

Multitask Learning For Different Subword Segmentations In Neural Machine Translation Tejas Srinivasan, Ramon Sanabria, Florian Metze Language Technologies Institute Carnegie Mellon University, USA tsriniva, ramons, fmetze@cs.cmu.edu Abstract In Neural Machine Translation (NMT) the usage of sub- words and characters as source and target units offers a sim- ple and flexible solution for translation of rare and unseen words. However, selecting the optimal subword segmenta- tion involves a trade-off between expressiveness and flexi- bility, and is language and dataset-dependent. We present Block Multitask Learning (BMTL), a novel NMT architec- ture that predicts multiple targets of different granularities simultaneously, removing the need to search for the optimal segmentation strategy. Our multi-task model exhibits im- provements of up to 1.7 BLEU points on each decoder over single-task baseline models with the same number of param- eters on datasets from two language pairs of IWSLT15 and one from IWSLT19. The multiple hypotheses generated at different granularities can be combined as a post-processing step to give better translations, which improves over hypoth- esis combination from baseline models while using substan- tially fewer parameters. 1. Introduction Neural Machine Translation (NMT) [1, 2, 3] provides a sim- ple, end-to-end framework for translating text from one lan- guage to another. NMT approaches have largely outper- formed and replaced previous statistical translation methods. Traditionally, NMT systems used words as source and tar- get units, which have three main disadvantages. First, word- based models are unable to translate rare and out of vocab- ulary (OOV) words in the source language. Second, they can not produce unseen target words, such as morphologi- cal variants of observed words (e.g., deriving realistic from real). Third, they have to handle large source and target language vocabularies (i.e., large look-up matrices), which makes them less scalable in term of computation and mem- ory. A large vocabulary also implies data sparsity where the number of tokens is not balanced. A common solution for the problems mentioned above is to perform word segmentation. The Byte-Pair Encod- ing (BPE) algorithm [4] groups units together according to their frequency. By presetting the desired vocabulary size, the BPE algorithm generates a segmentation of the data by �I �flew �on �Air �Force �Tw o �for �eight �years �. Subword1k Subword300 Shared Encoder Subword10k PROJ BiLSTM … BiLSTM n <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="hP+6LrUf2d3tZaldqaQvEKMXyw=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+ gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odBu3wMYA6nMFXEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit> <latexit sha1_base64="NtnmDLJ1NgRz18QxoCqIYjLd6rc=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5 +6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPOMxspUP9I0wzDhIy1jKTg5q6kH1Zpf9xdimxCsoAYrNQbVr/4wFXmCmoTi1vYCP6Ow4IakUDgr93OLGRcTPsKeQ80TtGxGHTGLp0zZHFq3NHEFu7vFwVPrJ0mkbuZcBrb9Wxu/pf1copvw0LqLCfUYvlRnCtGKZtvzYbSoCA1dcCFkW5WJsbcEGum7IrIVhfeRPa1/XArwdNH0pwDhdwBQHcwB08QANaIADhBd7g3XvyXr2PZV1b3q3M/gj7/MHwtmLnw=</lat exit> <latexit sha1_base64="NtnmDLJ1NgRz18QxoCqIYjLd6rc=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5 +6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPOMxspUP9I0wzDhIy1jKTg5q6kH1Zpf9xdimxCsoAYrNQbVr/4wFXmCmoTi1vYCP6Ow4IakUDgr93OLGRcTPsKeQ80TtGxGHTGLp0zZHFq3NHEFu7vFwVPrJ0mkbuZcBrb9Wxu/pf1copvw0LqLCfUYvlRnCtGKZtvzYbSoCA1dcCFkW5WJsbcEGum7IrIVhfeRPa1/XArwdNH0pwDhdwBQHcwB08QANaIADhBd7g3XvyXr2PZV1b3q3M/gj7/MHwtmLnw=</lat exit> <latexit sha1_base64="ck1/v/JpOJELH0iIdgPbhaWErXY=">AB6HicbVBNT8JAEJ3iF+IX6tHLRmLibRe9Ej04hESCyTQkO0yhZXtndmpCGX+DFg8Z49Sd589+4QA8KvmSl/dmMjMvTAXxnW/ndLG5tb 2Tnm3srd/cHhUPT5p6yRTDH2WiER1Q6pRcIm+4UZgN1VI41BgJ5zczf3OEyrNE/lgpikGMR1JHnFGjZVaclCtuXV3AbJOvILUoEBzUP3qDxOWxSgNE1TrnuemJsipMpwJnFX6mcaUsgkdYc9SWPUQb4dEYurDIkUaJsSUMW6u+JnMZaT+PQdsbUjPWqNxf/83qZiW6CnMs0MyjZclGUCWISMv+aDLlCZsTUEsoUt7cSNqaKMmOzqdgQvNWX10n7qu65da/l1hq3RxlOINzuAQPrqEB9AEHxgPMrvDmPzovz7nwsW0tOMXMKf+B8/gDV0 Yzu</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> <latexit sha1_base64="IqW2N36QktBwntGQavLhE1DLrE=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1v bO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1FipKQflilt1FyDrxMtJBXI0BuWv/jBmaYTSMEG17nluYvyMKsOZwFmpn2pMKJvQEfYslTRC7WeLQ2fkwipDEsbKljRkof6eyGik9TQKbGdEzVivenPxP6+XmvDGz7hMUoOSLReFqSAmJvOvyZArZEZMLaFMcXsrYWOqKDM2m5INwVt9eZ20r6qeW/Wa15X6bR5HEc7gHC7BgxrU4R4a0AIGCM/wCm/Oo/PivDsfy9aCk8+cwh84nz/XE Yzy</latexit> 1 <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="hP+6LrUf2d3tZaldqaQvEKMXyw=">AB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i7+ gsTLXTzQrMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GS7KDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RtRxzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D7odBu3wMYA6nMFXEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit> <latexit sha1_base64="2yIKfhykAKv0MhUqDiS4SywkL1Y=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPO MxspUP9I0wzDhIy1jKTg5qxkMqjW/7i/ENiFYQ1WagyqX/1hKvIENQnFre0FfkZhwQ1JoXBW7ucWMy4mfIQ9h5onaMNiMeiMXTpnyOLUuKOJLdzfLwqeWDtNIncz4TS269nc/C/r5RTfhoXUWU6oxfKjOFeMUjbfmg2lQUFq6oALI92sTIy54YJcN2VXQrC+8ia0r+uBXw+aPpTgHC7gCgK4gTt4gAa0QADC7zBu/fkvXofy7q2vFVvZ/BH3ucPaQSLYg=</latexit> <latexit sha1_base64="2yIKfhykAKv0MhUqDiS4SywkL1Y=">AB3XicbZBLSwMxFIXv+Ky1anXrJlgEV2XGjS4FNy5bsA9oh5J7SxmcyQ3BHK0F/gxoUi/i13/hvTx0JbDwQ+zknIvSfKlLTk+9/e1vbO7t5+6aB8WDk6PqmeVto2zY3AlkhVaroRt6ikxhZJUtjNDPIkUtiJvfzvPO MxspUP9I0wzDhIy1jKTg5qxkMqjW/7i/ENiFYQ1WagyqX/1hKvIENQnFre0FfkZhwQ1JoXBW7ucWMy4mfIQ9h5onaMNiMeiMXTpnyOLUuKOJLdzfLwqeWDtNIncz4TS269nc/C/r5RTfhoXUWU6oxfKjOFeMUjbfmg2lQUFq6oALI92sTIy54YJcN2VXQrC+8ia0r+uBXw+aPpTgHC7gCgK4gTt4gAa0QADC7zBu/fkvXofy7q2vFVvZ/BH3ucPaQSLYg=</latexit> <latexit sha1_base64="1enTr8x+dPZYKeA2su7IqvnUdLk=">AB6HicbVBNT8JAEJ3iF+IX6tHLRmLibRe9Ej04hESCyTQkO0yhZXtndmpCGX+DFg8Z49Sd589+4QA8KvmSl/dmMjMvTAXxnW/ndLG5tb2Tnm3srd/cHhUPT5p6yRTDH2WiER1Q6pRcIm+4UZgN1VI41BgJ5zczf3 OEyrNE/lgpikGMR1JHnFGjZVa3qBac+vuAmSdeAWpQYHmoPrVHyYsi1EaJqjWPc9NTZBTZTgTOKv0M40pZRM6wp6lksaog3x6IxcWGVIokTZkoYs1N8TOY21nsah7YypGetVby7+5/UyE90EOZdpZlCy5aIoE8QkZP41GXKFzIipJZQpbm8lbEwVZcZmU7EheKsvr5P2Vd1z617LrTVuizjKcAbncAkeXEMD7qEJPjBAeIZXeHMenRfn3flYtpacYuYU/sD5/AF5XYyx</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> <latexit sha1_base64="l9eImvYcFOKpzEDji/n9jPDeWb8=">AB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm/GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dz vPKHSPJYPZpqgH9GR5CFn1Fip6Q3KFbfqLkDWiZeTCuRoDMpf/WHM0gilYJq3fPcxPgZVYzgbNSP9WYUDahI+xZKmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGNn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUWZsNiUbgrf68jpX1U9t+o1ryv12zyOIpzBOVyCBzWowz0oAUMEJ7hFd6cR+fFeXc+lq0FJ585hT9wPn8Aep2MtQ=</latexit> UniLSTM decoder PROJ Subword10k PROJ Unit-specific CE Loss Unit-specific CE Loss Unit-specific CE Loss … … UniLSTM decoder UniLSTM decoder �I �declare �resum �ed �the �session �on �Friday �I �declare �resumed �the …

Content truncated to 500 words