Extrinsic Evaluation

This paper reports a preliminary study addressing two challenges in measuring the effectiveness of information extrac- • tion (IE) technology: Developing a methodology for ex- trinsic evaluation of IE; and, …...

In this paper we present an approach to large-scale coreference resolution for an ample set of human languages, with a par- ticular emphasis on time performance and precision. One of …...

Topic models have great potential for help- ing users understand document corpora. This potential is stymied by their purely un- supervised nature, which often leads to top- ics that are …...

We present a model for detecting user dis- engagement during spoken dialogue interac- tions. Intrinsic evaluation of our model (i.e., with respect to a gold standard) yields results on par …...

AbstractThis paper presents an approach of automatic annotation of sentences with dependency structures. The approach builds on the idea of cross-lingual dependency projection. The presented method of acquiring dependency trees …...

Word embeddings — distributed word representations that can be learned from unlabelled data — have been shown to have high utility in many natural language processing applications. In this paper, …...

AbstractThe exchangeability assumption in topic models like Latent Dirichlet Allocation (LDA) often results in inferring inconsistent topics for the words of text spans like noun-phrases, which are usually expected to …...

AbstractTopic models jointly learn topics and document-level topic distribution. Extrinsic evaluation of topic models tends to focus exclusively on topic-level evaluation, e.g. by assessing the coherence of topics. We demonstrate …...

AbstractWe propose to use question answering (QA) data from Web forums to train chat-bots from scratch, i.e., without dialog data. First, we extract pairs of question and answer sentences from …...

AbstractMeasuring domain relevance of data and identifying or selecting well-fit domain data for machine translation (MT) is a well-studied topic, but denoising is not yet. Denoising is concerned with a …...

AbstractFollowing the recent success of word embeddings, it has been argued that there is no such thing as an ideal representation for words, as different models tend to capture divergent …...

AbstractAttempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. We present a new approach based on scaled-up qualitative analysis of …...

AbstractWe propose a simple unsupervised method for extracting pseudo-parallel monolingual sentence pairs from comparable corpora representative of two different text styles, such as news articles and scientific papers. Our approach …...

AbstractOut-of-vocabulary words are still a challenge in cross-lingual Natural Language Processing tasks, for which transliteration from source to target language or script is one of the solutions. In this study, …...

AbstractThis paper presents the first ever comprehensive evaluation of different types of word embeddings for Sinhala language. Three standard word embedding models, namely, Word2Vec (both Skipgram and CBOW), FastText, and …...

AbstractWe present IndoBERTweet, the first large-scale pretrained model for Indonesian Twitter that is trained by extending a monolingually-trained Indonesian BERT model with additive domain-specific vocabulary. We focus in particular on …...

AbstractThis paper presents a simple but effective method to build sentiment lexicons for the three Mainland Scandinavian languages: Danish, Norwegian and Swedish. This method benefits from the English Sentiwordnet and …...

AbstractWord embedding models have become commonplace in a wide range of NLP applications. In order to train and use the best possible models, accurate evaluation is needed. For extrinsic evaluation …...