Элемент Жауап Теориясы
Related Papers
Building an Evaluation Scale using Item Response Theory
Evaluation of NLP methods requires test- ing against a previously vetted gold-standard test set and reporting standard metrics (ac- curacy/precision/recall/F1). The current as- sumption is that all items in a …...
Building an English Vocabulary Knowledge Dataset of Japanese English-as-a-Second-Language Learners Using Crowdsourcing
We introduce a freely available dataset for analyzing the English vocabulary of English-as-a-second language (ESL) learners. While ESL vocabulary tests have been extensively studied, few of the results have been …...
Learning Latent Parameters without Human Response Patterns: Item Response Theory with Artificial Crowds
AbstractIncorporating Item Response Theory (IRT) into NLP tasks can provide valuable information about model performance and behavior. Traditionally, IRT models are learned using human response pattern (RP) data, presenting a …...
Identifying Annotator Bias: A new IRT-based method for bias identification
AbstractA basic step in any annotation effort is the measurement of the Inter Annotator Agreement (IAA). An important factor that can affect the IAA is the presence of annotator bias. …...
Word Discriminations for Vocabulary Inventory Prediction
AbstractThe aim of vocabulary inventory prediction is to predict a learner’s whole vocabulary based on a limited sample of query words. This paper approaches the problem starting from the 2-parameter …...