Part-of-Speech Tagging for Historical English

Proceedings of NAACL-HLT 2016, pages 1318–1328, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Part-of-Speech Tagging for Historical English Yi Yang and Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology Atlanta, GA 30308 {yiyang+jacobe}@gatech.edu Abstract As more historical texts are digitized, there is interest in applying natural language pro- cessing tools to these archives. However, the performance of these tools is often unsatisfac- tory, due to language change and genre dif- ferences. Spelling normalization heuristics are the dominant solution for dealing with histori- cal texts, but this approach fails to account for changes in usage and vocabulary. In this em- pirical paper, we assess the capability of do- main adaptation techniques to cope with his- torical texts, focusing on the classic bench- mark task of part-of-speech tagging. We eval- uate several domain adaptation methods on the task of tagging Early Modern English and Modern British English texts in the Penn Cor- pora of Historical English. We demonstrate that the Feature Embedding method for unsu- pervised domain adaptation outperforms word embeddings and Brown clusters, showing the importance of embedding the entire feature space, rather than just individual words. Fea- ture Embeddings also give better performance than spelling normalization, but the combina- tion of the two methods is better still, yielding a 5% raw improvement in tagging accuracy on Early Modern English texts. 1 Introduction There is growing interest in applying natural lan- guage processing (NLP) techniques to historical texts (Piotrowski, 2012), with applications in infor- mation retrieval (Dougherty, 2010; Jurish, 2011), linguistics (Baron et al., 2009; Rayson et al., 2007), and the digital humanities (Hendrickx et al., 2011; Original: and drewe vnto hym all ryottours & wylde dys- posed persones Normalization: and drew unto him all ryottours & wild disposed persons Figure 1: An example sentence from Early Modern English and its VARD normalization. Muralidharan and Hearst, 2013; Pettersson and Nivre, 2011). However, these texts differ from con- temporary training corpora in a number of linguistic respects, including the lexicon (Giusti et al., 2007), morphology (Borin and Forsberg, 2008), and syn- tax (Eumeridou et al., 2004). This imposes signif- icant challenges for modern NLP tools: for exam- ple, the accuracy of the CLAWS part-of-speech Tag- ger (Garside and Smith, 1997) drops from 97% on the British National Corpus to 82% on Early Mod- ern English texts (Rayson et al., 2007). There are two main approaches that could improve the accu- racy of NLP systems on historical texts: normaliza- tion and domain adaptation. Normalization Spelling normalization (also called canonicalization) involves mapping histor- ical spellings to their canonical forms in modern languages, thus bridging the gap between contem- porary training corpora and target historical texts. Figure 1 shows one historical sentence and its normalization by VARD (Baron and Rayson, 2008). Rayson et al. (2007) report an increase of about 3% accuracy on adaptation of POS tagging from Modern English texts to Early Modern English texts if the target texts were automatically normalized by the VARD system. However, normalization …

Content truncated to 500 words