Authorship Attribution and Verification with Many Authors and Limited Data

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 513–520 Manchester, August 2008 Authorship Attribution and Verification with Many Authors and Limited Data Kim Luyckx and Walter Daelemans CNTS Language Technology Group University of Antwerp Prinsstraat 13, 2000 Antwerp, Belgium {kim.luyckx,walter.daelemans}@ua.ac.be Abstract Most studies in statistical or machine learning based authorship attribution focus on two or a few authors. This leads to an overestimation of the importance of the features extracted from the training data and found to be discriminating for these small sets of authors. Most studies also use sizes of training data that are unreal- istic for situations in which stylometry is applied (e.g., forensics), and thereby over- estimate the accuracy of their approach in these situations. A more realistic interpre- tation of the task is as an authorship ver- ification problem that we approximate by pooling data from many different authors as negative examples. In this paper, we show, on the basis of a new corpus with 145 authors, what the effect is of many authors on feature selection and learning, and show robustness of a memory-based learning approach in doing authorship at- tribution and verification with many au- thors and limited training data when com- pared to eager learning methods such as SVMs and maximum entropy learning. 1 Introduction In traditional studies on authorship attribution, the focus is on small sets of authors. Trying to classify an unseen text as being written by one of two or of a few authors is a relatively simple task, which c⃝ Kim Luyckx & Walter Daelemans, 2008. Li- censed under the Creative Commons Attribution- Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. in most cases can be solved with high reliabil- ity and accuracies over 95%. An early statistical study by Mosteller and Wallace (1964) adopted distributions of function words as a discriminat- ing feature to settle the disputed authorship of the Federalist Papers between three candidate authors (Alexander Hamilton, James Madison, and John Jay). The advantage of distributions of function words and syntactic features is that they are not under the author’s conscious control, and there- fore provide good clues for authorship (Holmes, 1994). Frequencies of rewrite rules (Baayen et al., 1996), n-grams of syntactic labels from par- tial parsing (Hirst and Feiguina, 2007), n-grams of parts-of-speech (Diederich et al., 2000), function words (Miranda Garc´ıa and Calle Mart´ın, 2007), and functional lexical features (Argamon et al., 2007) have all been claimed to be reliable markers of style. There is of course a difference between claims about types of features and claims about in- dividual features of that type. E.g., it may be cor- rect to claim that distributions of function words are important markers of author identity, but the distribution of a particular function word, while useful to distinguish between one particular pair of authors, may be irrelevant when comparing an- other pair of authors. The field of authorship attribution is however dominated by studies potentially overestimating the importance of these specific predictive features …

Content truncated to 500 words