Көрнекі ақпарат
Related Papers
Using Visual Information to Predict Lexical Preference
Most NLP systems make predictions based solely on linguistic (textual or spo- ken) input. We show how to use visual information to make better linguistic pre- dictions. We focus on …...
Is an Image Worth More than a Thousand Words? On the Fine-Grain Semantic Differences between Visual and Linguistic Representations
AbstractHuman concept representations are often grounded with visual information, yet some aspects of meaning cannot be visually represented or are better described with language. Thus, vision and language provide complementary …...
What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues
AbstractGrounding a pronoun to a visual object it refers to requires complex reasoning from various information sources, especially in conversational scenarios. For example, when people in a conversation talk about …...
Encoding Lexico-Semantic Knowledge using Ensembles of Feature Maps from Deep Convolutional Neural Networks
AbstractSemantic models derived from visual information have helped to overcome some of the limitations of solely text-based distributional semantic models. Researchers have demonstrated that text and image-based representations encode complementary …...
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation
AbstractThis paper addresses the problem of simultaneous machine translation (SiMT) by exploring two main concepts: (a) adaptive policies to learn a good trade-off between high translation quality and low latency; …...
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
AbstractWith the increasing popularity of posting multimodal messages online, many recent studies have been carried out utilizing both textual and visual information for multi-modal sarcasm detection. In this paper, we …...
Learning the Effects of Physical Actions in a Multi-modal Environment
AbstractLarge Language Models (LLMs) handle physical commonsense information inadequately. As a result of being trained in a disembodied setting, LLMs often fail to predict an action’s outcome in a given …...