Визуальные особенности

We explore the problem of resolving the second person English pronoun you in multi-party dialogue, using a combination of linguistic and visual features. First, we distinguish generic and referential uses, …...

Most NLP systems make predictions based solely on linguistic (textual or spo- ken) input. We show how to use visual information to make better linguistic pre- dictions. We focus on …...

________________________________________________________________________________________________________________ Mapping of the spectral space to the visual speech space for French vowels cued in Cued Speech In this paper, we present a statistical method based on GMM modeling …...

Determining the reading level of children’s lit- erature is an important task for providing edu- cators and parents with an appropriate reading trajectory through a curriculum. Automating this process has …...

Recent investigations into grounded models of language have shown that holistic views of language and perception can provide higher performance than independent views. In this work, we improve a two-dimensional …...

Information in visually rich formats such as PDF and HTML is often conveyed by a combination of textual and visual features. In particular, genres such as marketing flyers and info-graphics …...

We investigate the role that geometric, tex- tual and visual features play in the task of predicting a preposition that links two visual entities depicted in an image. The task …...

In this paper we focus on a new problem of event coreference resolution across televi- sion news videos. Based on the observa- tion that the contents from multiple data modalities …...

A large part of human communication in- volves referring to entities in the world and often these entities are objects that are visu- ally present for the interlocutors. A system …...

AbstractInstructional videos get high-traffic on video sharing platforms, and prior work suggests that providing time-stamped, subtask annotations (e.g., “heat the oil in the pan”) improves user experiences. However, current automatic …...

AbstractThis paper studies keyphrase extraction in real-world scenarios where documents are from diverse domains and have variant content quality. We curate and release OpenKP, a large scale open domain keyphrase …...

AbstractIn this paper, we address the task of news-image captioning, which generates a description of an image given the image and its article body as input. This task is more …...

AbstractOpen-domain Keyphrase extraction (KPE) on the Web is a fundamental yet complex NLP task with a wide range of practical applications within the field of Information Retrieval. In contrast to …...

AbstractWe introduce a new neural network architecture, Multimodal Neural Graph Memory Networks (MN-GMN), for visual question answering. The MN-GMN uses graph structure with different region features as node attributes and …...

AbstractWeb search is an essential way for humans to obtain information, but it’s still a great challenge for machines to understand the contents of web pages. In this paper, we …...

AbstractAttention based methods for image-text generation often focus on visual features individually, while ignoring relationship information among image features that provides important guidance for generating sentences. To alleviate this issue, …...

AbstractCharts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer …...

AbstractMulti-modal neural machine translation (MNMT) aims to improve textual level machine translation performance in the presence of text-related images. Most of the previous works on MNMT focus on multi-modal fusion …...