Мультимодальность
Related Papers
Le son de tes lèvres : corrélats électrophysiologiques de la perception audio-haptique de la parole. (The sound of your lips: electrophysiological correlates of audio-haptic speech perception) [in French]
_______________________________________________________________________________________________ The sound of you The sound of you The sound of you The sound of yourrrr lips lips lips lips: electrophysiological correl : electrophysiological correl : electrophysiological correl : …...
Enhancing the AI2 Diagrams Dataset Using Rhetorical Structure Theory
This paper describes ongoing work on a multimodal resource based on the Allen Institute AI2 Diagrams (AI2D) dataset, which contains nearly 5000 grade-school level science diagrams that have been annotated …...
FEIDEGGER: A Multi-modal Corpus of Fashion Images and Descriptions in German
The availability of multi-modal datasets that pair images and textual descriptions of their content has been a crucial driver in progress of various text-image tasks such as automatic captioning and …...
Procedural Reasoning Networks for Understanding Multimodal Procedures
AbstractThis paper addresses the problem of comprehending procedural commonsense knowledge. This is a challenging task as it requires identifying key entities, keeping track of their state changes, and understanding temporal …...
The YiTrans Speech Translation System for IWSLT 2022 Offline Shared Task
AbstractThis paper describes the submission of our end-to-end YiTrans speech translation system for the IWSLT 2022 offline task, which translates from English audio to German, Chinese, and Japanese. The YiTrans …...
Augmenting pre-trained language models with audio feature embedding for argumentation mining in political debates
AbstractThe integration of multimodality in natural language processing (NLP) tasks seeks to exploit the complementary information contained in two or more modalities, such as text, audio and video. This paper …...