Entropy
Related Papers
Predication of Meaning of Bisyllabic Chinese Compound Words Using Back Propagation Neural Network
A three layer back propagation neural net is set up to study the functional dependency between the semantic class of a bisyllabic Chinese word and that of its two constituent …...
Improving Language Model Size Reduction using Better Pruning Criteria
Reducing language model (LM) size is a critical issue when applying a LM to realistic applications which have memory constraints. In this paper, three measures are studied for the purpose …...
Variation of Entropy and Parse Trees of Sentences as a Function of the Sentence Number
In this paper we explore the variation of sentences as a function of the sentence number. We demonstrate that while the entropy of the sentence increases with the sentence number, …...
Robust Target Speaker Tracking in Broadcast TV Streams
This paper addresses the problem of audio change detection and speaker tracking in broadcast TV streams. A two-pass audio change detection algorithm, which includes detection of the potential change boundaries …...
OntoNotes: Corpus Cleanup of Mistaken Agreement Using Word Sense Disambiguation
Annotated corpora are only useful if their annotations are consistent. Most large-scale annotation efforts take special measures to reconcile inter-annotator disagreement. To date, however, no-one has investigated how to automatically …...
Prototype-based Active Learning for Lemmatization
Annotation of training data for machine learning is often a laborious and costly process. In Active Learning (AL), crite- ria are investigated that allow ordering the unannotated data in such …...
Investigations into the Crandem Approach to Word Recognition
We suggest improvements to a previously pro- posed framework for integrating Conditional Random Fields and Hidden Markov Models, dubbed a Crandem system (2009). The pre- vious authors’ work suggested that …...
Unsupervised Model Adaptation using Information-Theoretic Criterion
In this paper we propose a novel general framework for unsupervised model adapta- tion. Our method is based on entropy which has been used previously as a regularizer in semi-supervised …...
Hello, Who is Calling?: Can Words Reveal the Social Nature of Conversations?
This study aims to infer the social nature of conversations from their content automat- ically. To place this work in context, our moti- vation stems from the need to understand …...
Automatic Discovery of Adposition Typology
Natural languages (NL) can be classified as prepositional or postpositional based on the order of the noun phrase and the adposition. Categorizing a language by its adposition typology helps in …...
Mapping CPA Patterns onto OntoNotes Senses
AbstractIn this paper we present an alignment experiment between patterns of verb use discovered by Corpus Pattern Analysis (CPA; Hanks 2004, 2008, 2012) and verb senses in OntoNotes (ON; Hovy …...
Comparing Apples to Apple: The Effects of Stemmers on Topic Models
AbstractRule-based stemmers such as the Porter stemmer are frequently used to preprocess English corpora for topic modeling. In this work, we train and evaluate topic models on a variety of …...
German in Flux: Detecting Metaphoric Change via Word Entropy
AbstractThis paper explores the information-theoretic measure entropy to detect metaphoric change, transferring ideas from hypernym detection to research on language change. We build the first diachronic test set for German …...
On the Derivational Entropy of Left-to-Right Probabilistic Finite-State Automata and Hidden Markov Models
AbstractProbabilistic finite-state automata are a formalism that is widely used in many problems of automatic speech recognition and natural language processing. Probabilistic finite-state automata are closely related to other finite-state …...
Speakers Fill Lexical Semantic Gaps with Context
AbstractLexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more efficient. If ambiguous words cannot be disambiguated from context, however, this …...
Textual Data Augmentation for Efficient Active Learning on Tiny Datasets
AbstractIn this paper we propose a novel data augmentation approach where guided outputs of a language generation model, e.g. GPT-2, when labeled, can improve the performance of text classifiers through …...
Efficient Computation of Expectations under Spanning Tree Distributions
AbstractWe give a general framework for inference in spanning tree models. We propose unified algorithms for the important cases of first-order expectations and second-order expectations in edge-factored, non-projective spanning-tree models. …...
Reanalyzing the Most Probable Sentence Problem: A Case Study in Explicating the Role of Entropy in Algorithmic Complexity
AbstractWhen working with problems in natural language processing, we can find ourselves in situations where the traditional measurements of descriptive complexity are ineffective at describing the behaviour of our algorithms. …...
Improving Aspect Sentiment Quad Prediction via Template-Order Data Augmentation
AbstractRecently, aspect sentiment quad prediction (ASQP) has become a popular task in the field of aspect-level sentiment analysis. Previous work utilizes a predefined template to paraphrase the original sentence into …...
Stop Measuring Calibration When Humans Disagree
AbstractCalibration is a popular framework to evaluate whether a classifier knows when it does not know - i.e., its predictive probabilities are a good indication of how likely a prediction …...
Generalizing Backpropagation for Gradient-Based Interpretability
AbstractMany popular feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model’s output with respect to its inputs. While these methods can indicate which input …...