Redefining part-of-speech classes with distributional semantic models

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pages 115–125, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Redefining part-of-speech classes with distributional semantic models Andrey Kutuzov Department of Informatics University of Oslo andreku@ifi.uio.no Erik Velldal Department of Informatics University of Oslo erikve@ifi.uio.no Lilja Øvrelid Department of Informatics University of Oslo liljao@ifi.uio.no Abstract This paper studies how word embeddings trained on the British National Corpus in- teract with part of speech boundaries. Our work targets the Universal PoS tag set, which is currently actively being used for annotation of a range of languages. We ex- periment with training classifiers for pre- dicting PoS tags for words based on their embeddings. The results show that the in- formation about PoS affiliation contained in the distributional vectors allows us to discover groups of words with distribu- tional patterns that differ from other words of the same part of speech. This data often reveals hidden inconsisten- cies of the annotation process or guide- lines. At the same time, it supports the notion of ‘soft’ or ‘graded’ part of speech affiliations. Finally, we show that infor- mation about PoS is distributed among dozens of vector components, not limited to only one or two features. 1 Introduction Parts of speech (PoS) are useful abstractions, but still abstractions. Boundaries between them in nat- ural languages are flexible. Sometimes, large open classes of words are situated on the verge between several parts of speech: for example, participles in English are in many respects both verbs and ad- jectives. In other cases, closed word classes ‘inter- sect’, e.g., it is often difficult to tell a determiner from a possessive pronoun. As Houston (1985) puts it, ‘Grammatical categories exist along a con- tinuum which does not exhibit sharp boundaries between the categories’. When annotating natural language texts for parts of speech, the choice of a PoS tag in many ways depends on the human annotators them- selves, but also on the quality of linguistic con- ventions behind the division into different word classes. That is why there have been several at- tempts to refine the definitions of parts of speech and to make them more empirically grounded, based on corpora of real texts: see, among others, the seminal work of Biber et al. (1999). The aim of such attempts is to identify clusters of words occurring naturally and corresponding to what we usually call ‘parts of speech’. One of the main distance metrics that can be used in detecting such clusters is a distance between distributional fea- tures of words (their contexts in a reference train- ing corpus). In this paper, we test this approach using pre- dictive models developed in the field of distribu- tional semantics. Recent achievements in training distributional models of language using machine learning allow for robust representations of nat- ural language semantics created in a completely unsupervised way, using only large corpora of raw text. Relations between dense word vectors (em- beddings) in the resulting vector space are …

Content truncated to 500 words