Knowing the Author by the Company His Words Keep

Knowing the Author by the Company His Words Keep Armin Hoenen1, Niko Schenk2 1CEDIFOR, 2Applied Computational Linguistics Lab (ACoLi) Goethe University, Frankfurt am Main, Germany {hoenen,nschenk}@em.uni-frankfurt.de Abstract In this paper, we analyze relationships between word pairs and evaluate their idiosyncratic properties in the applied context of authorship attribution. Specifically, on three literary corpora we optimize word pair features for information gain which reflect word similarity as measured by word embeddings. We analyze the quality of the most informative features in terms of word type relation (a comparison of different constellations of function and content words), similarity, and relatedness. Results point to the extraordinary role of function words within the authorship attribution task being extended to their pairwise relational patterns. Similarity of content words is likewise among the most informative features. From a cognitive perspective, we conclude that both relationship types reflect short distance connections in the human brain, which is highly indicative of an individual writing style. Keywords: Word pairs, word relations, authorship attribution 1. Introduction In traditional linguistics, there exists a famous saying that one should know a word by the company it keeps (Firth, 1957), which informally describes the meaning of a partic- ular word by the context in which it occurs. In this paper, we investigate sets of frequently appearing similar words and their relations. To this end, we make use of word em- beddings as a substitute and ground our work on related applications of distributed word representations and their applications (Mikolov et al., 2013a; Mikolov et al., 2013b; Mikolov et al., 2013c), which allows us to investigate rela- tionships and even associations between words. Word em- beddings have become popular recently and have been suc- cessfully applied to a variety of NLP tasks. Generally, word embeddings are believed to capture distributional similar- ity with an implication to semantic similarity. Kiela et al. (2015), for example, investigate relatedness and similarity and find embeddings to be specializable for both phenom- ena. In our work, we focus our word pair-based methodology on the particular domain of authorship attribution. From an interpretation perspective, we conjecture that our pro- posed method allows to inspect not only single words in isolation and ask what kind of word choices may character- ize certain authors (Marsden et al., 2013), but also to focus on properties of latent syntactic and semantic relations be- tween words. 2. Word Relations & Human Cognition In this study a special focus is laid on word pairs and their relations. Those can be understood as edges in a wider net- work and thus represent a first step towards modeling true semantic connections: Although finding a direct neurologi- cal correlate for words is a rather complex and controversial endeavour, many methods and theories even in psychology implicitly or explicitly operate with such networks. In com- putational linguistics, for instance, word nets have become a popular way of rationalizing relationships between con- cepts manifested through words. Especially theories on the mental lexicon belong to this sphere, cf. Elman (2004); Libben and …

Content truncated to 500 words