MultimodalMachineTranslation
Related Papers
Distilling Translations with Visual Awareness
AbstractPrevious work on multimodal machine translation has shown that visual information is only needed in very specific cases, for example in the presence of ambiguous words where the textual context …...
Probing the Need for Visual Context in Multimodal Machine Translation
AbstractCurrent work on multimodal machine translation (MMT) has suggested that the visual modality is either unnecessary or only marginally beneficial. We posit that this is a consequence of the very …...
Multimodal Transformer for Multimodal Machine Translation
AbstractMultimodal Machine Translation (MMT) aims to introduce information from other modality, generally static images, to improve the translation quality. Previous works propose various incorporation methods, but most of them do …...
BERTGen: Multi-task Generation through BERT
AbstractWe present BERTGen, a novel, generative, decoder-only model which extends BERT by fusing multimodal and multilingual pre-trained models VL-BERT and M-BERT, respectively. BERTGen is auto-regressively trained for language generation tasks, …...
Cross-lingual Visual Pre-training for Multimodal Machine Translation
AbstractPre-trained language models have been shown to improve performance in many natural language tasks substantially. Although the early focus of such models was single language pre-training, recent advances have resulted …...
Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models
AbstractMultimodal machine translation (MMT) systems have been shown to outperform their text-only neural machine translation (NMT) counterparts when visual context is available. However, recent studies have also shown that the …...
Distill The Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation
AbstractPast works on multimodal machine translation (MMT) elevate bilingual setup by incorporating additional aligned vision information. However, an image-must requirement of the multimodal dataset largely hinders MMT’s development — namely …...
Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective
AbstractMultimodal machine translation (MMT) aims to improve translation quality by equipping the source sentence with its corresponding image. Despite the promising performance, MMT models still suffer the problem of input …...
Neural Machine Translation with Phrase-Level Universal Visual Representations
AbstractMultimodal machine translation (MMT) aims to improve neural machine translation (NMT) with additional visual information, but most existing MMT methods require paired input of source sentence and image, which makes …...
VISA: An Ambiguous Subtitles Dataset for Visual Scene-aware Machine Translation
AbstractExisting multimodal machine translation (MMT) datasets consist of images and video captions or general subtitles which rarely contain linguistic ambiguity, making visual information not so effective to generate appropriate translations. …...
A Filtering Approach to Object Region Detection in Multimodal Machine Translation
AbstractRecent studies in Multimodal Machine Translation (MMT) have explored the use of visual information in a multimodal setting to analyze its redundancy with textual information. The aim of this work …...