Word Frequency
Related Papers
Extension of Zipf’s Law to Words and Phrases
Zipf’s law states that the frequency of word tokens in a large corpus of natural language is inversely proportional to the rank. The law is investigated for two languages English …...
Trend-based Document Clustering for Sensitive and Stable Topic Detection
. The ability to detect new topics and track them is important given the huge amounts of documents. This paper introduces a trend-based document clustering algorithm for analyzing them. Its …...
How Specialized are Specialized Corpora? Behavioral Evaluation of Corpus Representativeness for Maltese.
AbstractIn this paper we bring to light a novel intersection between corpus linguistics and behavioral data that can be employed as an evaluation metric for resources for low-density languages, drawing …...
Creating a massively parallel Bible corpus
AbstractWe present our ongoing effort to create a massively parallel Bible corpus. While an ever-increasing number of Bible translations is available in electronic form on the internet, there is no …...
A Turkish Database for Psycholinguistic Studies Based on Frequency, Age of Acquisition, and Imageability
AbstractThis study primarily aims to build a Turkish psycholinguistic database including three variables: word frequency, age of acquisition (AoA), and imageability, where AoA and imageability information are limited to nouns. …...
A large-scale study of the effects of word frequency and predictability in naturalistic reading
AbstractA number of psycholinguistic studies have factorially manipulated words’ contextual predictabilities and corpus frequencies and shown separable effects of each on measures of human sentence processing, a pattern which has …...
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation
AbstractSequence-to-sequence models for open-domain dialogue generation tend to favor generic, uninformative responses. Past work has focused on word frequency-based approaches to improving specificity, such as penalizing responses with only common …...
Serial Recall Effects in Neural Language Modeling
AbstractSerial recall experiments study the ability of humans to recall words in the order in which they occurred. The following serial recall effects are generally investigated in studies with humans: …...
UTFPR at SemEval-2021 Task 1: Complexity Prediction by Combining BERT Vectors and Classic Features
AbstractWe describe the UTFPR systems submitted to the Lexical Complexity Prediction shared task of SemEval 2021. They perform complexity prediction by combining classic features, such as word frequency, n-gram frequency, …...
Word Discriminations for Vocabulary Inventory Prediction
AbstractThe aim of vocabulary inventory prediction is to predict a learner’s whole vocabulary based on a limited sample of query words. This paper approaches the problem starting from the 2-parameter …...
NyLLex: A Novel Resource of Swedish Words Annotated with Reading Proficiency Level
AbstractWhat makes a text easy to read or not, depends on a variety of factors. One of the most prominent is, however, if the text contains easy, and avoids difficult, …...
Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps
AbstractIR models using a pretrained language model significantly outperform lexical approaches like BM25. In particular, SPLADE, which encodes texts to sparse vectors, is an effective model for practical use because …...
Word Acquisition in Neural Language Models
AbstractWe investigate how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory (Fenson et …...