Visual Grounding
Related Papers
Grounded Textual Entailment
AbstractCapturing semantic relations between sentences, such as entailment, is a long-standing challenge for computational semantics. Logic-based models analyse entailment in terms of possible worlds (interpretations, or situations) where a premise …...
Knowledge Supports Visual Language Grounding: A Case Study on Colour Terms
AbstractIn human cognition, world knowledge supports the perception of object colours: knowing that trees are typically green helps to perceive their colour in certain contexts. We go beyond previous studies …...
Visual Grounding Annotation of Recipe Flow Graph
AbstractIn this paper, we provide a dataset that gives visual grounding annotations to recipe flow graphs. A recipe flow graph is a representation of the cooking workflow, which is designed …...
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers
AbstractMirroring the success of masked language models, vision-and-language counterparts like VILBERT, LXMERT and UNITER have achieved state of the art performance on a variety of multimodal discriminative tasks like visual …...
Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering
AbstractVisual question answering (VQA) is challenging not only because the model has to handle multi-modal information, but also because it is just so hard to collect sufficient training examples — …...
Inducing Semantic Roles Without Syntax
Visual dialogue is a challenging task since it needs to answer a series of coherent ques- tions on the basis of understanding the vi- sual environment. Previous studies focus on …...
OCID-Ref: A 3D Robotic Dataset With Embodied Language For Clutter Scene Grounding
AbstractTo effectively apply robots in working environments and assist humans, it is essential to develop and evaluate how visual grounding (VG) can affect machine performance on occluded objects. However, current …...
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency
AbstractRecent research in Visual Question Answering (VQA) has revealed state-of-the-art models to be inconsistent in their understanding of the world - they answer seemingly difficult questions requiring reasoning correctly but …...
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
AbstractIn this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or …...
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
AbstractLarge-scale pre-trained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks. This paper presents …...