Articles

This paper aims to compare different reg- ularization strategies to address a com- mon phenomenon, severe overfitting, in embedding-based neural networks for NLP. We chose two widely studied neu- ral models and tasks as our testbed. We tried several frequently applied or newly proposed regularization strategies, including penalizing weights (embeddings …...

When applying machine learning to prob- lems in NLP, there are many choices to make about how to represent input texts. They can have a big effect on perfor- mance, but they are often uninteresting to researchers or practitioners who simply need a module that performs well. We ap- ply …...

Hyper-parameter optimization is an im- portant problem in natural language pro- cessing (NLP) and machine learning. Re- cently, a group of studies has focused on using sequential Bayesian Optimization to solve this problem, which aims to reduce the number of iterations and trials required during the optimization process. In this …...

The neural network joint model (NNJM), which augments the neural network lan- guage model (NNLM) with an m-word source context window, has achieved large gains in machine translation accuracy, but also has problems with high normalization cost when using large vocabularies. Train- ing the NNJM with noise-contrastive es- timation (NCE), …...

Recent work in neural machine translation has shown promising performance, but the most effective architectures do not scale naturally to large vocabulary sizes. We propose and compare three variable-length encoding schemes that represent a large vocabulary corpus using a much smaller vocabulary with no loss in information. Common words are …...

Events and their coreference offer use- ful semantic and discourse resources. We show that the semantic and dis- course aspects of events interact with each other. However, traditional approaches ad- dressed event extraction and event coref- erence resolution either separately or se- quentially, which limits their interactions. This paper proposes …...

When translating between two languages that differ in their degree of morpholog- ical synthesis, syntactic structures in one language may be realized as morphologi- cal structures in the other, and SMT mod- els need a mechanism to learn such trans- lations. Prior work has used morpheme splitting with flat representations …...

In the last two years, there has been a surge of word embedding algorithms and research on them. However, evaluation has mostly been carried out on a narrow set of tasks, mainly word similarity/relatedness and word relation similarity and on a single language, namely English. We propose an approach to …...

We present a higher-order inference sys- tem based on a formal compositional semantics and the wide-coverage CCG parser. We develop an improved method to bridge between the parser and seman- tic composition. The system is evaluated on the FraCaS test suite. In contrast to the widely held view that higher-order …...

Unsupervisedly learned word vectors have proven to provide exceptionally effective features in many NLP tasks. Most common intrinsic evaluations of vector quality mea- sure correlation with similarity judgments. However, these often correlate poorly with how well the learned representations per- form as features in downstream evaluation tasks. We present QVEC—a …...

Large-scale Knowledge Bases (such as NELL, Yago, Freebase, etc.) are often sparse, i.e., a large number of valid rela- tions between existing entities are missing. Recent research have addressed this prob- lem by augmenting the KB graph with ad- ditional edges mined from a large text cor- pus while keeping …...

Usernames are ubiquitous on the Internet, and they are often suggestive of user de- mographics. This work looks at the de- gree to which gender and language can be inferred from a username alone by making use of unsupervised morphology induction to decompose usernames into sub-units. Experimental results on the …...

Machine Translation (MT) has advanced in re- cent years to produce better translations for clients’ specific domains, and sophisticated tools allow professional translators to obtain translations according to their prior edits. We suggest that MT should be further personal- ized to the end-user level – the receiver or the author …...

This paper addresses the question of how language use affects community reaction to comments in online discussion forums, and the relative importance of the mes- sage vs. the messenger. A new comment ranking task is proposed based on com- munity annotated karma in Reddit discus- sions, which controls for topic …...

We present a multilingual corpus of Wikipedia and Twitter texts annotated with FRAMENET 1.5 semantic frames in nine different languages, as well as a novel technique for weakly supervised cross-lingual frame-semantic parsing. Our approach only assumes the existence of linked, comparable source and target lan- guage corpora (e.g., Wikipedia) and …...

We describe the WIKIQA dataset, a new publicly available set of question and sen- tence pairs, collected and annotated for re- search on open-domain question answer- ing. Most previous work on answer sen- tence selection focuses on a dataset cre- ated using the TREC-QA data, which includes editor-generated questions and …...

Much of what we understand from text is not explicitly stated. Rather, the reader uses his/her knowledge to fill in gaps and create a coherent, mental picture or “scene” depicting what text appears to convey. The scene constitutes an under- standing of the text, and can be used to an- …...

We demonstrate the advantage of special- izing semantic word embeddings for either similarity or relatedness. We compare two variants of retrofitting and a joint-learning approach, and find that all three yield spe- cialized semantic spaces that capture hu- man intuitions regarding similarity and re- latedness better than unspecialized spaces. We …...

The conferences ACL (Association for Computational Linguistics) and EMNLP (Empirical Methods in Natural Language Processing) rank among the premier venues that track the research develop- ments in Natural Language Processing and Computational Linguistics. In this paper, we present a study on the research pa- pers of approximately two decades from …...

Direct content analysis reveals important details about movies including those of gender representations and potential bi- ases. We investigate the differences be- tween male and female character depic- tions in movies, based on patterns of lan- guage used. Specifically, we use an au- tomatically generated lexicon of linguis- tic norms …...