Learning from Within? Comparing PoS Tagging Approaches for Historical Text

Learning from Within? Comparing PoS Tagging Approaches for Historical Text Sarah Schulz, Jonas Kuhn Institute for Natural Language Processing (IMS) University of Stuttgart Pfaffenwaldring 5b, 70569 Stuttgart firstname.lastname@ims.uni-stuttgart.de Abstract In this paper, we investigate unsupervised and semi-supervised methods for part-of-speech (PoS) tagging in the context of historical German text. We locate our research in the context of Digital Humanities where the non-canonical nature of text causes issues facing an Natural Language Processing world in which tools are mainly trained on standard data. Data deviating from the norm requires tools adjusted to this data. We explore to which extend the availability of such training material and resources related to it influences the accuracy of PoS tagging. We investigate a variety of algorithms including neural nets, conditional random fields and self-learning techniques in order to find the best-fitted approach to tackle data sparsity. Although methods using resources from related languages outperform weakly supervised methods using just a few training examples, we can still reach a promising accuracy with methods abstaining additional resources. Keywords: PoS tagging, historical language, self-taught learning, neural nets, stacking, Digital Humanities, low resource languages 1. Introduction PoS (part-of-speech) tagging is a preprocessing step which is indispensable for many Natural Language Processing (NLP) tasks and needs to be done with high accuracy to ensure success in the subsequent task. Therefore, it is a well understood field offering a variety of techniques suit- able for different languages. Schmid (1995) reports PoS tagging accuracy of 97.5% on German newspaper text. However, most of these approaches use a large number of training examples. Even if it comes to unsupervised meth- ods, the unlabeled amount of data has to be satisfactorily large. Yet, there are scenarios in which neither labeled nor unlabeled data is sufficiently available. Digital Humanities (DH) is a field of research that is devel- oping fast. It holds its very own kind of challenges and sci- entific issues both interesting for Humanities and Computer Science. One of the problems for NLP is the non-canonical nature of text especially found in projects dealing with his- torical text. These texts deviate from text that has been the main focus of NLP so far by lacking standardized orthog- raphy and grammar. This can often lead to a decrease in performance of tools trained on standard text (Melero et al., 2012; Eisenstein, 2013). Data needed for training dedicated tools for these texts is often not available. However, in case the non-canonical text is related to another language, e.g., the modern stage of the language, we can exploit this relat- edness to facilitate tagging. To illustrate this, we investigate PoS tagging of a unique late Middle High German (MHG) text in the transition pe- riod between MHG and Early New High German (ENHG). This leads to a text with mixed features of two historical stages of German. In this paper, we investigate PoS tagging for historical texts sharing a lot of the challenges of PoS tagging for low re- sourced languages. We do this by means …

Content truncated to 500 words