Articles

There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering ques- tions about images. These tasks have fo- cused on literal descriptions of the image. To move beyond the literal, we choose to explore …...

Linguistics studies have shown that action verbs often denote some Change of State (CoS) as the result of an action. However, the causality of action verbs and its poten- tial connection with the physical world has not been systematically explored. To ad- dress this limitation, this paper presents a study …...

We study the problem of automatically building hypernym taxonomies from tex- tual and visual data. Previous works in taxonomy induction generally ignore the increasingly prominent visual data, which encode important perceptual semantics. Instead, we propose a probabilistic model for taxonomy induction by jointly leverag- ing text and images. To avoid …...

Automatically generating a natural lan- guage description of an image is a fun- damental problem in artificial intelligence. This task involves both computer vision and natural language processing and is called “image caption generation.” Re- search on image caption generation has typically focused on taking in an image and generating …...

This paper describes the first robust ap- proach to automatically labeling clauses with their situation entity type (Smith, 2003), capturing aspectual phenomena at the clause level which are relevant for interpreting both semantics at the clause level and discourse structure. Previous work on this task used a small data set …...

We present a pairwise context-sensitive Autoencoder for computing text pair sim- ilarity. Our model encodes input text into context-sensitive representations and uses them to compute similarity between text pairs. Our model outperforms the state-of-the-art models in two semantic re- trieval tasks and a contextual word simi- larity task. For retrieval, …...

Quotation detection is the task of locating spans of quoted speech in text. The state of the art treats this problem as a sequence labeling task and employs linear-chain con- ditional random fields. We question the ef- ficacy of this choice: The Markov assump- tion in the model prohibits it …...

Word pairs, which are one of the most eas- ily accessible features between two text segments, have been proven to be very useful for detecting the discourse relations held between text segments. However, be- cause of the data sparsity problem, the performance achieved by using word pair features is limited. …...

Neural machine translation (NMT) mod- els typically operate with a fixed vocabu- lary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this pa- per, we introduce a simpler and more ef- fective approach, making the NMT model …...

Activities and events in our lives are struc- tural, be it a vacation, a camping trip, or a wedding. While individual details vary, there are characteristic patterns that are specific to each of these scenarios. For ex- ample, a wedding typically consists of a sequence of events such as walking …...

Discriminative translation models utiliz- ing source context have been shown to help statistical machine translation perfor- mance. We propose a novel extension of this work using target context information. Surprisingly, we show that this model can be efficiently integrated directly in the de- coding process. Our approach scales to large …...

This paper addresses the problem of speech act recognition in written asyn- chronous conversations (e.g., fora, emails). We propose a class of conditional structured models defined over arbitrary graph structures to capture the conversa- tional dependencies between sentences. Our models use sentence representations encoded by a long short term memory …...

We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is ca- pable of optimizing model parameters di- rectly with respect to arbitrary evaluation metrics, which are not necessarily differ- entiable. Experiments show that our ap- proach achieves significant improvements over maximum …...

Recent work has shown that simple vector subtraction over word embeddings is surpris- ingly effective at capturing different lexical relations, despite lacking explicit supervision. Prior work has evaluated this intriguing result using a word analogy prediction formulation and hand-selected relations, but the generality of the finding over a broader range …...

Despite interest in using cross-lingual knowledge to learn word embeddings for various tasks, a systematic comparison of the possible approaches is lacking in the literature. We perform an extensive eval- uation of four popular approaches of in- ducing cross-lingual embeddings, each re- quiring a different form of supervision, on four …...

Languages with rich inflectional morphol- ogy exhibit lexical data sparsity, since the word used to express a given concept will vary with the syntactic context. For in- stance, each count noun in Czech has 12 forms (where English uses only singular and plural). Even in large corpora, we are un- …...

Advances in transfer learning have let go the limitations of traditional supervised machine learning algorithms for being de- pendent on annotated training data for training new models for every new do- main. However, several applications en- counter scenarios where models need to transfer/adapt across domains when the la- bel sets …...

We address an important problem in sequence-to-sequence (Seq2Seq) learning referred to as copying, in which cer- tain segments in the input sequence are selectively replicated in the output se- quence. A similar phenomenon is ob- servable in human language communica- tion. For example, humans tend to re- peat entity names …...

This paper introduces a novel architec- ture for reinforcement learning with deep neural networks designed to handle state and action spaces characterized by natural language, as found in text-based games. Termed a deep reinforcement relevance network (DRRN), the architecture repre- sents action and state spaces with sepa- rate embedding vectors, …...

Structured prediction from bandit feed- back describes a learning scenario where instead of having access to a gold standard structure, a learner only receives partial feedback in form of the loss value of a pre- dicted structure. We present new learning objectives and algorithms for this inter- active scenario, focusing …...