Мультимодальный Трансформер
Related Papers
Multimodal Transformer for Unaligned Multimodal Language Sequences
AbstractHuman language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors. However, two major challenges in modeling such multimodal human language time-series data exist: …...
Multiˆ2OIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT
AbstractIn this paper, we propose Multi2OIE, which performs open information extraction (open IE) by combining BERT with multi-head attention. Our model is a sequence-labeling system with an efficient and effective …...
Attention as Grounding: Exploring Textual and Cross-Modal Attention on Entities and Relations in Language-and-Vision Transformer
AbstractWe explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked …...
xGQA: Cross-Lingual Visual Question Answering
AbstractRecent advances in multimodal vision and language modeling have predominantly focused on the English language, mostly due to the lack of multilingual multimodal datasets to steer modeling efforts. In this …...