Active Learning

We explore how active learning with Sup- port Vector Machines works well for a non-trivial task in natural language pro- cessing. We use Japanese word segmenta- tion as a test …...

Active learning (AL) promises to reduce the cost of annotating labeled datasets for trainable human language technologies. Contrary to expectations, when creating labeled training material for HPSG parse selection and …...

Supervised estimation methods are widely seen as being superior to semi and fully unsuper- vised methods. However, supervised methods crucially rely upon training sets that need to be manually annotated. …...

In this paper, we propose a multi-criteria- based active learning approach and effec- tively apply it to named entity recognition. Active learning targets to minimize the human annotation efforts by …...

We report on an active learning experi- ment for named entity recognition in the astronomy domain. Active learning has been shown to reduce the amount of la- belled data required …...

This paper presents an adaptive learning framework for Phonetic Similarity Modeling (PSM) that supports the automatic construction of transliteration lexicons. The learning algorithm starts with minimum prior knowledge about machine …...

In this paper we present an active learn- ing approach used to create an annotated corpus of literal and nonliteral usages of verbs. The model uses nearly unsu- pervised word-sense …...

We consider the impact Active Learning (AL) has on effective and efficient text cor- pus annotation, and report on reduction rates for annotation efforts ranging up until 72%. We also …...

When a word sense disambiguation (WSD) system is trained on one domain but ap- plied to a different domain, a drop in ac- curacy is frequently observed. This high- lights …...

This paper addresses two issues of active learning. Firstly, to solve a problem of uncertainty sampling that it often fails by selecting outliers, this paper presents a new selective sampling …...

In this paper, we address the problem of knowing when to stop the process of active learning. We propose a new statistical learning approach, called minimum expected error strategy, to …...

In this paper, we address the issue of de- ciding when to stop active learning for building a labeled training corpus. Firstly, this paper presents a new stopping crite- rion, …...

Active learning is a proven method for re- ducing the cost of creating the training sets that are necessary for statistical NLP. How- ever, there has been little work on …...

The accuracy of current word sense disam- biguation (WSD) systems is affected by the fine-grained sense inventory of WordNet as well as a lack of training examples. Using the WSD …...

Methods that learn from prior informa- tion about input features such as general- ized expectation (GE) have been used to train accurate models with very little ef- fort. In this …...

Annotation of training data for machine learning is often a laborious and costly process. In Active Learning (AL), crite- ria are investigated that allow ordering the unannotated data in such …...

AbstractLarge scale parallel data generation for new language pairs requires intensive human effort and availability of experts. It becomes immensely difficult and costly to provide Statistical Machine Translation (SMT) systems …...

AbstractThis paper describes how we built a dependency Treebank for questions. The questions for the Treebank were drawn from questions from the TREC 10 QA task and from Yahoo! Answers. …...

AbstractWe describe the re-annotation of selected types of named entities (persons, organizations, locations) from the Muc7 corpus. The focus of this annotation initiative is on recording the time needed for …...

Active learning has been applied to dif- ferent NLP tasks, with the aim of limit- ing the amount of time and cost for human annotation. Most studies on active learn- …...

We explore how to improve machine trans- lation systems by adding more translation data in situations where we already have substantial resources. The main challenge is how to buck the …...

Confidence-Weighted linear classifiers (CW) and its successors were shown to perform well on binary and multiclass NLP prob- lems. In this paper we extend the CW ap- proach for sequence …...

We present a general method for incorporating an “expert” model into a Statistical Machine Transla- tion (SMT) system, in order to improve its perfor- mance on a particular “area of …...

AbstractExpert human input can contribute in various ways to facilitate automatic annotation of natural language text. For example, a part-of-speech tagger can be trained on labeled input provided offline by …...

We investigate active learning methods for Japanese dependency parsing. We propose active learning methods of using partial dependency relations in a given sentence for parsing and evaluate their effective- ness …...

Supervised classification needs large amounts of annotated training data that is expensive to create. Two approaches that reduce the cost of annotation are active learning and crowd- sourcing. However, these …...

We present an empirical study of instance se- lection techniques for machine translation. In an active learning setting, instance selection minimizes the human effort by identifying the most informative sentences …...

We report on our ongoing work in devel- oping the Irish Dependency Treebank, de- scribe the results of two Inter-annotator Agreement (IAA) studies, demonstrate improvements in annotation consistency which have …...

We present an active learning method for coreference resolution that is novel in three re- spects. (i) It uses bootstrapped neighborhood pooling, which ensures a class-balanced pool even though gold …...

Active learning is a promising way for sentiment classification to reduce the annotation cost. In this paper, we focus on the imbalanced class distribution scenario for sentiment classification, wherein the …...

Translation needs have greatly increased during the last years. In many situa- tions, text to be translated constitutes an unbounded stream of data that grows con- tinually with time. An …...

We describe the results of our submissions to the WMT13 Shared Task on Quality Estimation (subtasks 1.1 and 1.3). Our submissions use the framework of Gaus- sian Processes to investigate …...

AbstractThis paper presents a new active learning technique for machine translation based on quality estimation of automatically translated sentences. It uses an error-driven strategy, i.e., it assumes that the more …...

Active learning (AL) consists of asking human annotators to annotate automatically selected data that are assumed to bring the most bene- fit in the creation of a classifier. AL allows …...

Active learning (AL) has been proven ef- fective to reduce human annotation ef- forts in NLP. However, previous studies on AL are limited to applications in a single language. This …...

Post-editing the output of a statistical machine translation (SMT) system to obtain high-quality translation has become an increasingly common application of SMT, which henceforth we re- fer to as post-editing-based …...

Coreference is a core nlp problem. How- ever, newswire data, the primary source of existing coreference data, lack the rich- ness necessary to truly solve coreference. We present a new …...

Different from traditional active learning based on sentence-wise full annotation (FA), this paper proposes active learning with dependency-wise partial annotation (PA) as a finer-grained unit for dependency parsing. At each …...

We seek to address the lack of labeled data (and high cost of annotation) for textual entailment in some domains. To that end, we first create (for experimental purposes) an …...

Effective text classification requires experts to annotate data with labels; these training data are time-consuming and expensive to obtain. If you know what labels you want, active learning can reduce …...

The ability to compute an accurate re- ward function is essential for optimising a dialogue policy via reinforcement learn- ing. In real-world applications, using ex- plicit user feedback as the …...

Active learning is a framework that makes it possible to efficiently train statistical models by selecting informative examples from a pool of unlabeled data. Previous work has found this framework …...

AbstractActive learning (AL) is often used in corpus construction (CC) for selecting “informative” documents for annotation. This is ideal for focusing annotation efforts when all documents cannot be annotated, but …...

This paper describes our approach to the Se- mEval 2016 task 4, “Sentiment Analysis in Twitter”, where we participated in subtask A. Our system relies on AlchemyAPI and Senti- WordNet …...

This study investigates the use of unsu- pervised word embeddings and sequence features for sample representation in an active learning framework built to extract clinical concepts from clinical free text. …...

AbstractIn this paper, we propose an extractive multi-document summarization (MDS) system using joint optimization and active learning for content selection grounded in user feedback. Our method interactively obtains user feedback …...

AbstractActive learning aims to select a small subset of data for annotation such that a classifier learned on the data is highly accurate. This is usually done using heuristic selection …...

AbstractRecognizing mentions of Adverse Drug Reactions (ADR) in social media is challenging: ADR mentions are context-dependent and include long, varied and unconventional descriptions as compared to more formal medical symptom …...

AbstractWe study the application of active learning techniques to the translation of unbounded data streams via interactive neural machine translation. The main idea is to select, from an unbounded stream …...

This paper is a short empirical study of the performance of centrality and classification based iterative term set expansion methods for distributional semantic models. Iterative term set expansion is an …...

AbstractFundamental to several knowledge-centric applications is the need to identify named entities from their textual mentions. However, entities lack a unique representation and their mentions can differ greatly. These variations …...

AbstractTraditional active learning (AL) methods for machine translation (MT) rely on heuristics. However, these heuristics are limited when the characteristics of the MT problem change due to e.g. the language …...

AbstractMost work on part-of-speech (POS) tagging is focused on high resource languages, or examines low-resource and active learning settings through simulated studies. We evaluate POS tagging techniques on an actual …...

AbstractThis paper presents RuSentiment, a new dataset for sentiment analysis of social media posts in Russian, and a new set of comprehensive annotation guidelines that are extensible to other languages. …...

AbstractActive learning (AL) is a technique for reducing manual annotation effort during the annotation of training data for machine learning classifiers. For NLP tasks, pool-based and stream-based sampling techniques have …...

AbstractNeural sequence-to-sequence models have been successfully applied to text compression. However, these models were trained on huge automatically induced parallel corpora, which are only available for a few domains and …...

AbstractEntity resolution (ER) is the task of identifying different representations of the same real-world entities across databases. It is a key step for knowledge base creation and text mining. Recent …...

AbstractThe exploding cost and time needed for data labeling and model training are bottlenecks for training DNN models on large datasets. Identifying smaller representative data samples with strategies like active …...

AbstractWhile high quality gold standard annotated corpora are crucial for most tasks in natural language processing, many annotated corpora published in recent years, created by annotators or tools, contains noisy …...

AbstractNatural language understanding (NLU) aims at identifying user intent and extracting semantic slots. This requires sufficient annotating data to get considerable performance in real-world situations. Active learning (AL) has been …...

AbstractImitation learning algorithms provide state-of-the-art results on many structured prediction tasks by learning near-optimal search policies. Such algorithms assume training-time access to an expert that can provide the optimal action …...

AbstractActive learning is an efficient approach for mitigating data dependency when training neural machine translation (NMT) models. In this paper, we explore new training frameworks by incorporating active learning into …...

AbstractActive learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strategy is contingent on the classification model. For instance, uncertainty sampling …...

AbstractExisting approaches to active learning maximize the system performance by sampling unlabeled instances for annotation that yield the most efficient training. However, when active learning is integrated with an end-user …...

AbstractRecently, leveraging pre-trained Transformer based language models in down stream, task specific models has advanced state of the art results in natural language understanding tasks. However, only a little research …...

AbstractStructured representations of entity names are useful for many entity-related tasks such as entity normalization and variant generation. Learning the implicit structured representations of entity names without context and external …...

AbstractMany pairwise classification tasks, such as paraphrase detection and open-domain question answering, naturally have extreme label imbalance (e.g., 99.99% of examples are negatives). In contrast, many recent datasets heuristically choose …...

AbstractDeep learning models are the current State-of-the-art methodologies towards many real-world problems. However, they need a substantial amount of labeled data to be trained appropriately. Acquiring labeled data can be …...

AbstractActive learning is an important technique for low-resource sequence labeling tasks. However, current active sequence labeling methods use the queried samples alone in each iteration, which is an inefficient way …...

AbstractIn this paper we propose a novel data augmentation approach where guided outputs of a language generation model, e.g. GPT-2, when labeled, can improve the performance of text classifiers through …...

AbstractThis paper presents an active learning approach that aims to reduce the human effort required during the annotation of natural language corpora composed of entities and semantic relations. Our approach …...

AbstractAnnotating training data for sequence tagging of texts is usually very time-consuming. Recent advances in transfer learning for natural language processing in conjunction with active learning open the possibility to …...

AbstractWhile deep learning is a powerful tool for natural language processing (NLP) problems, successful solutions to these problems rely heavily on large amounts of annotated samples. However, manually annotating data …...

AbstractEntity Alignment (EA) aims to match equivalent entities across different Knowledge Graphs (KGs) and is an essential step of KG fusion. Current mainstream methods – neural EA models – rely …...

AbstractNamed entity recognition (NER) is one of the major tasks in natural language processing. A named entity is often a word or expression that bears a valuable piece of information, …...

AbstractInteractive-predictive translation is a collaborative iterative process and where human translators produce translations with the help of machine translation (MT) systems interactively. Various sampling techniques in active learning (AL) exist …...

AbstractActive learning promises to alleviate the massive data needs of supervised machine learning: it has successfully improved sample efficiency by an order of magnitude on traditional tasks like topic classification …...

AbstractActive learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost. This is now an essential tool for building low-resource syntactic analyzers such as …...

AbstractActive Learning (AL) has been successfully applied to Deep Learning in order to drastically reduce the amount of data required to achieve high performance. Previous works have shown that lightweight …...

AbstractOpen pit mines left many regions worldwide inhospitable or uninhabitable. Many sites are left behind in a hazardous or contaminated state, show remnants of waste, or have other restrictions imposed …...

AbstractThis paper describes our systems for negation detection and time expression recognition in SemEval 2021 Task 10, Source-Free Domain Adaptation for Semantic Processing. We show that self-training, active learning and …...

AbstractThe aim of vocabulary inventory prediction is to predict a learner’s whole vocabulary based on a limited sample of query words. This paper approaches the problem starting from the 2-parameter …...

AbstractIn this work, we provide a literature review of active learning (AL) for its applications in natural language processing (NLP). In addition to a fine-grained categorization of query strategies, we …...

AbstractNeural coreference resolution models trained on one dataset may not transfer to new, low-resource domains. Active learning mitigates this problem by sampling a small subset of data for annotators to …...

AbstractThe automation of extracting argument structures faces a pair of challenges on (1) encoding long-term contexts to facilitate comprehensive understanding, and (2) improving data efficiency since constructing high-quality argument structures …...

AbstractLarge scale, multi-label text datasets with high numbers of different classes are expensive to annotate, even more so if they deal with domain specific language. In this work, we aim …...

AbstractMulti-task learning, in which several tasks are jointly learned by a single model, allows NLP models to share information from multiple annotations and may facilitate better predictions when the tasks …...

AbstractDetecting disclosures of individuals’ employment status on social media can provide valuable information to match job seekers with suitable vacancies, offer social protection, or measure labor market flows. However, identifying …...

AbstractActive learning is the iterative construction of a classification model through targeted labeling, enabling significant labeling cost savings. As most research on active learning has been carried out before transformer-based …...

AbstractState-of-the-art classification and regression models are often not well calibrated, and cannot reliably provide uncertainty estimates, limiting their utility in safety-critical applications such as clinical decision-making. While recent work has …...

AbstractWe study the out-of-distribution generalization of active learning that adaptively selects samples for annotation in learning the decision boundary of classification. Our empirical study finds that increasingly annotating seen samples …...

AbstractIn recent years, active learning has been successfully applied to an array of NLP tasks. However, prior work often assumes that training and test data are drawn from the same …...

AbstractIn this paper we investigate the application of active learning to semantic role labeling (SRL) using Bayesian Active Learning by Disagreement (BALD). Our new predicate-focused selection method quickly improves efficiency …...

AbstractOne significant obstacle to the successful application of machine learning to real-world data is that of labeling: it is often prohibitively expensive to pay an ethical amount for the human …...

AbstractWhile transformer-based systems have enabled greater accuracies with fewer training examples, data acquisition obstacles still persist for rare-class tasks – when the class label is very infrequent (e.g. < 5% …...

AbstractWe present our submission to SemEval-2023 Task 10: Explainable Detection of Online Sexism (EDOS). We address all three tasks: Task A consists of identifying whether a post is sexist. If …...