Articles

In the last several years, neural network models have significantly improved accu- racy in a number of NLP tasks. How- ever, one serious drawback that has im- peded their adoption in production sys- tems is the slow runtime speed of neu- ral network models compared to alternate models, such as …...

A wide range of applications, from social media to scientific literature analysis, in- volve graphs in which documents are con- nected by links. We introduce a topic model for link prediction based on the in- tuition that linked documents will tend to have similar topic distributions, integrat- ing a max-margin …...

Research on modeling time series text cor- pora has typically focused on predicting what text will come next, but less well studied is predicting when the next text event will occur. In this paper we ad- dress the latter case, framed as modeling continuous inter-arrival times under a log- Gaussian …...

When building spoken dialogue systems for a new domain, a major bottleneck is developing a spoken language understand- ing (SLU) module that handles the new domain’s terminology and semantic con- cepts. We propose a statistical SLU model that generalises to both previously unseen input words and previously unseen out- put …...

We investigate an extension of continuous online learning in recurrent neural network language models. The model keeps a sep- arate vector representation of the current unit of text being processed and adaptively adjusts it after each prediction. The initial experiments give promising results, indi- cating that the method is able …...

In this paper we explore a POS tagging ap- plication of neural architectures that can infer word representations from the raw character stream. It relies on two mod- elling stages that are jointly learnt: convolutional network that infers a worda representation directly from the character stream, followed by a prediction …...

This paper discusses the use of factoriza- tion techniques in distributional semantic models. We focus on a method for re- distributing the weight of latent variables, which has previously been shown to im- prove the performance of distributional se- mantic models. However, this result has not been replicated and remains …...

Despite the convexity of structured max- margin objectives (Taskar et al., 2004; Tsochantaridis et al., 2004), the many ways to optimize them are not equally ef- fective in practice. We compare a range of online optimization methods over a vari- ety of structured NLP tasks (coreference, summarization, parsing, etc) and …...

IBM Model 1 is a classical alignment model. Of the first generation word-based SMT models, it was the only such model with a concave objective function. For concave optimization problems like IBM Model 1, we have guarantees on the con- vergence of optimization algorithms such as Expectation Maximization (EM). How- …...

We investigate the role that geometric, tex- tual and visual features play in the task of predicting a preposition that links two visual entities depicted in an image. The task is an important part of the subsequent process of generating image descriptions. We explore the prediction of prepositions for a …...

It is well-known that readers are less likely to fixate their gaze on closed class syn- tactic categories such as prepositions and pronouns. This paper investigates to what extent the syntactic category of a word in context can be predicted from gaze fea- tures obtained using eye-tracking equip- ment. If …...

: It has not been evaluated on empirical data and applied to any language tasks. We generate concrete models for this setting by developing algorithms to construct tensors and linear maps and instantiate the abstract parameters using empirical data. We then evaluate our concrete models against several experiments, both existing …...

Supervised word sense disambiguation (WSD) systems are usually the best per- forming systems when evaluated on stan- dard benchmarks. However, these systems need annotated training data to function properly. While there are some publicly available open source WSD systems, very few large annotated datasets are available to the research community. …...

Model selection (picking, for example, the feature set and the regularization strength) is crucial for building high-accuracy NLP models. In supervised learning, we can es- timate the accuracy of a model on a subset of the labeled data and choose the model with the highest accuracy. In contrast, here we …...

We present a compositional distributional semantic model which is an implementa- tion of the tensor-based framework of Co- ecke et al. (2011). It is an extended skip- gram model (Mikolov et al., 2013) which we apply to adjective-noun combinations, learning nouns as vectors and adjectives as matrices. We also propose …...

Text classification tasks suffer from curse of dimensionality due to large feature space. Short text data further exacerbates the prob- lem due to their sparse and noisy nature. Fea- ture selection thus becomes an important step in improving the classification performance. In this paper, we propose a novel feature se- …...

Using automatic measures such as labeled and unlabeled attachment scores is com- mon practice in dependency parser evalu- ation. In this paper, we examine whether these measures correlate with human judg- ments of overall parse quality. We ask lin- guists with experience in dependency an- notation to judge system outputs. …...

The emergence of user forums in elec- tronic news media has given rise to the proliferation of opinion manipulation trolls. Finding such trolls automatically is a hard task, as there is no easy way to recognize or even to define what they are; this also makes it hard to get …...

We propose a method that learns a cross- lingual projection of word representations from one language into another. Our method utilizes translatable context pairs as bonus terms of the objective function. In the experiments, our method outper- formed existing methods in three language pairs, (English, Spanish), (Japanese, Chi- nese) and …...

We present a transition-based arc-eager model to parse spinal trees, a dependency- based representation that includes phrase- structure information in the form of con- stituent spines assigned to tokens. As a main advantage, the arc-eager model can use a rich set of features combining depen- dency and constituent information, while …...