Unsupervised Instance-Based Part of Speech Induction Using Probable Substitutes

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2303–2313, Dublin, Ireland, August 23-29 2014. Unsupervised Instance-Based Part of Speech Induction Using Probable Substitutes Mehmet Ali Yatbaz Enis Rıfat Sert Deniz Yuret Koc¸ University Artificial Intelligence Laboratory, ˙Istanbul {myatbaz,esert,dyuret}@ku.edu.tr Abstract We develop an instance (token) based extension of the state of the art word (type) based part-of- speech induction system introduced in (Yatbaz et al., 2012). Each word instance is represented by a feature vector that combines information from the target word and probable substitutes sampled from an n-gram model representing its context. Modeling ambiguity using an instance based model does not lead to significant gains in overall accuracy in part-of-speech tagging be- cause most words in running text are used in their most frequent class (e.g. 93.69% in the Penn Treebank). However it is important to model ambiguity because most frequent words are ambiguous and not modeling them correctly may negatively affect upstream tasks. Our main contribution is to show that an instance based model can achieve significantly higher accuracy on ambiguous words at the cost of a slight degradation on unambiguous ones, maintaining a compa- rable overall accuracy. On the Penn Treebank, the overall many-to-one accuracy of the system is within 1% of the state-of-the-art (80%), while on highly ambiguous words it is up to 70% better. On multilingual experiments our results are significantly better than or comparable to the best published word or instance based systems on 15 out of 19 corpora in 15 languages. The vector representations for words used in our system are available for download for further experiments. 1 Introduction Unsupervised part-of-speech (POS) induction aims to classify words into syntactic categories using un- labeled, plain text input. The problem of induction is important for studying under-resourced languages that lack labeled corpora and high quality dictionaries. It is also essential in modeling child language acquisition because every child manages to induce syntactic categories without access to labeled sen- tences, labeled prototypes, or dictionary constraints (Ambridge and Lieven, 2011). Categories induced from data may point to shortcomings or inconsistencies of hand-labeled categories as discussed in Sec- tion 4. Finally, the induced categories or the vector representations generated by the induction algorithms may improve natural language processing systems when used as additional features. Word-based POS induction systems classify different instances of a word in a single category (which we will refer to as the one-tag-per-word assumption). Instance-based systems classify each occurence of a word separately and can handle ambiguous words. Examples of word-based systems include ones that represent each word with the vector of neighboring words (context vectors) and cluster them (Sch¨utze, 1995; Lamar et al., 2010b; Lamar et al., 2010a), use a prototypical bi-tag HMM that assigns each word to a latent class (Brown et al., 1992; Clark, 2003), restrict a HMM based Pitman-Yor process to perform one-tag-per-word inference (Blunsom and Cohn, 2011), define a word-based Bayesian multinomial mixture model (Christodoulopoulos et al., 2011), or This work is licenced under a …

Content truncated to 500 words