Wikipedia Titles As Noun Tag Predictors

Wikipedia Titles as Noun Tag Predictors Armin Hoenen Goethe University Frankfurt Text Technology Lab Robert -Mayer Str. 10 60325 Frankfurt, Germany hoenen@em.uni-frankfurt.de Abstract In this paper, we investigate a covert labeling cue, namely the probability that a title (by example of the Wikipedia titles) is a noun. If this probability is very large, any list such as or comparable to the Wikipedia titles can be used as a reliable word-class (or part-of-speech tag) predictor or noun lexicon. This may be especially useful in the case of Low Resource Languages (LRL) where labeled data is lacking and putatively for Natural Language Processing (NLP) tasks such as Word Sense Disambiguation, Sentiment Analysis and Machine Translation. Profitting from the ease of digital publication on the web as opposed to print, LRL speaker communities produce resources such as Wikipedia and Wiktionary, which can be used for an assessment. We provide statistical evidence for a strong noun bias for the Wikipedia titles from 2 corpora (English, Persian) and a dictionary (Japanese) and for a typologically balanced set of 17 languages including LRLs. Additionally, we conduct a small experiment on predicting noun tags for out-of-vocabulary items in part-of-speech tagging for English. Keywords: Wikipedia, Wiktionary, Noun, Title 1. Introduction In linguistic literature, one finds that borrowed words are acknowledgedly overly likely to be nouns, (Haspelmath, 2008). Clues such as this one make it possible to use the information on one type of linguistic knowledge (borrow- ing) and to transfer it to another (word class). In this paper, we investigate, whether a large probability for being a noun likewise holds true for (single token) titles.1 We assume, that titles as found in the Wikipedia are often definition la- bels, which would help explain such a bias. However, the term definition must be used with caution since Wikipedia articles are not all necessarily definitions in the strict sense. For LRLs2 resources are lacking and state-of-the-art NLP is complicated, since supervised statistical algorithms typi- cally get performant with large quantities of labeled data for training. Yet speaker communities do produce written re- sources and make use of the proliferation and input ease of texts in the internet as opposed to the difficulties and costs of publishing a printed text. Especially crowd-sourced re- sources such as the Wikipedia have a growing body of texts even in LRLs. The question arises of how this text can be used to improve the resource situation of an LRL. Apart from compiling a corpus, which is the most obvious use of 1Multiple token titles constitute a considerable proportion of the Wikipedia titles, differing from language to language. Their amount ranged from 41 to 68 % with a mean of 54% as approx- imated by using the presence of the underscore. Generally it is believed here, that single token titles are more likely to be nouns and more likely to be definitions. Apart from this, using a mul- tiple token title as a label predictor for a single token requires to solve questions concerning overlap. The …

Content truncated to 500 words