Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3222–3233 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3222 Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation Julia Ive1, Andy Mingren Li1, Yishu Miao1, Ozan Caglayan1, Pranava Madhyastha1, Lucia Specia1,2,3 Imperial College London1, University of Sheffield2, ADAPT - Dublin City University3 j.ive@ic.ac.uk, andy.li16@imperial.ac.uk, y.miao20@imperial.ac.uk, o.caglayan@ic.ac.uk pranava@ic.ac.uk, l.specia@ic.ac.uk Abstract This paper addresses the problem of simulta- neous machine translation (SiMT) by explor- ing two main concepts: (a) adaptive policies to learn a good trade-off between high translation quality and low latency; and (b) visual infor- mation to support this process by providing ad- ditional (visual) contextual information which may be available before the textual input is pro- duced. For that, we propose a multimodal ap- proach to simultaneous machine translation us- ing reinforcement learning, with strategies to integrate visual and textual information in both the agent and the environment. We provide an exploration on how different types of vi- sual information and integration strategies af- fect the quality and latency of simultaneous translation models, and demonstrate that vi- sual cues lead to higher quality while keeping the latency low. 1 Introduction Research into automating real-time interpretation has explored deterministic and adaptive approaches to build policies that address the issue of trans- lation delay (Ryu et al., 2006; Cho and Esipova, 2016; Gu et al., 2017). In another recent devel- opment, the availability of multimodal data (such as visual information) has driven the community towards multimodal approaches for machine trans- lation (MMT) (Specia et al., 2016; Elliott et al., 2017; Barrault et al., 2018). Although determinis- tic policies have been recently explored for simul- taneous MMT (Caglayan et al., 2020; Imankulova et al., 2020), there are no studies regarding how multimodal information can be exploited to build flexible and adaptive policies for simultaneous ma- chine translation (SiMT). Applications of reinforcement learning (RL) for unimodal SiMT have highlighted the challenges for the agent to maintain good translation quality while learning an optimal translation path (i.e. a sequence of READ/WRITE decisions at every time step) (Grissom II et al., 2016; Gu et al., 2017; Aline- jad et al., 2018). Incomplete source information will have detri- mental effect especially in the cases where signifi- cant restructuring is needed while translating from one language to another. In addition, the lack of information generally leads to high variance during the training in the RL setup. We posit that multimodality in adaptive SiMT could help the agent by providing extra sig- nals, which would in turn improve training stability and thus the quality of the estimator and translation decoder. In this paper, we present the first exploration on multimodal RL approaches for the task of SiMT. As visual signals, we explore both image classi- fication features as well as visual concepts, which provide global image information and explicit ob- ject representations, respectively. For RL, we em- ploy the Policy Gradient method with a pre-trained neural machine translation model acting …

Content truncated to 500 words