Robust Morphological Tagging with Word Representations

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 526–536, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Robust Morphological Tagging with Word Representations Thomas M¨uller and Hinrich Sch¨utze Center for Information and Language Processing University of Munich, Germany muellets@cis.lmu.de Abstract We present a comparative investigation of word representations for part-of-speech (POS) and morphological tagging, focusing on sce- narios with considerable differences between training and test data where a robust approach is necessary. Instead of adapting the model towards a specific domain we aim to build a robust model across domains. We developed a test suite for robust tagging consisting of six languages and different domains. We find that representations similar to Brown clusters per- form best for POS tagging and that word rep- resentations based on linguistic morphological analyzers perform best for morphological tag- ging. 1 Introduction Most natural language processing (NLP) tasks can be better solved if a preprocessor tags each word in the natural language input with a label like “noun, singular” or “verb, past tense” that gives some in- dication of the syntactic role that the word plays in its context. The most common form of such pre- processing is POS tagging. However, for morpho- logically rich languages, a large subset of the lan- guages of the world, POS tagging in its original form – where labels are syntactic categories with little or no morphological information – does not make much sense. The reason is that POS and morpho- logical properties are mutually dependent, so solv- ing only one task or solving the tasks sequentially is inadequate. The most important dependence of this type is that POS can be read off morphology in many cases; e.g., the morphological suffix “-iste” is a reli- able indicator of the informal second person singu- lar preterite indicative form of a verb in Spanish. In what follows, we use the term “morphological tag- ging” to refer to “morphological and POS tagging” since morphological tags generally include POS in- formation. The importance of morphological tagging as part of the computational linguistics processing pipeline motivated us to conduct the research reported in this paper. The specific setting that we address is in- creasingly recognized as the setting in which most practical NLP takes place: We look at scenarios with considerable differences between the training data and the application data, i.e., between the data that the tagger is trained on and the data that it is ap- plied to. This type of scenario is frequent because of the great diversity and variability of natural language and because of the high cost of annotation – which makes it impossible to create large training sets for each new domain. For this reason, we address mor- phological tagging in a setting in which training and application data differ. The most common approach to this setting is domain adaptation. Domain adaptation has been demonstrated to have good performance in scenarios with differently distributed training/test data. …

Content truncated to 500 words