Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging

Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging Kyoko Sugisaki, Nicolas Wiedmer, Heiko Hausendorf German department, University of Zurich Sch¨onberggasse 9, 8001 Zurich, Switzerland {sugisaki, nicolas.wiedmer, heiko.hausendorf}@ds.uzh.ch Abstract In this paper, we present a corpus of over 11,000 holiday picture postcards written in German and Swiss German. The postcards have been collected for the purpose of text-linguistic investigations on the genre and its standardisation and variation over time. We discuss the processes and challenges of digitalisation, manual transcription, and manual annotation. In addition, we developed our own automatic text segmentation system and a part-of-speech tagger, since our texts often contain orthographic deviations, domain-specific structures such as fragments, subject-less sentences, interjections, discourse particles, and domain-specific formulaic communicative routines in salutation and greeting. In particular, we demonstrate that the CRF-based POS tagger could be boosted to a domain-specific text by adding a small amount of in-domain data. We showed that entropy-based training data sampling was competitive with random sampling in performing this task. The evaluation showed that our POS tagger achieved a F1 score of 0.93 (precision 0.94, recall 0.93), which outperformed a state-of-the-art POS tagger. Keywords: postcard corpus, POS tagging, German 1. Introduction In this paper, we report the construction of the language resource Ansichtskartenkorpus ([anko]), ‘picture postcard corpus’, containing over 11,000 holiday postcards written in Standard German and Swiss German. They were manu- ally transcribed and annotated with structural and discourse- related information, and then automatically annotated with text segmentation, lemma and part-of-speech (POS) infor- mation. We will first characterise the texts contained in the resource (Section 2), and then describe their manual transcription and annotation before outlining the development of a NLP toolkit for text segmentation and POS annotation (Section 3). 2. Data Source The holiday postcards were collected at the University of Zurich from 2009 to present day for the purpose of text- linguistic investigations on the genre and its standardisation and variation over time. The postcards included in our cor- pus were sent by post from people on holiday, mainly from Switzerland but also from Italy, Germany and other Euro- pean countries to their family, friends, colleagues and neigh- bours living in the German-speaking area of Switzerland. About 95% of the cards (11,760 cards) were written mainly in Standard German. The remaining part of the corpus is comprised of postcards written mainly in Swiss German. Although the postcards were dated from 1898 to 2016, the majority were written in the 1980s (22%) and 1990s (19%). On average, a post card contains 50 words, while individual post cards vary from one to 350 words. 3. Corpus Construction In this section, we describe the process of digitalisation, transcription, and annotation carried out manually and auto- matically to build the corpus of the collected postcards. 3.1. Overall Pipeline: From Digitalisation to XML with Linguistic Annotation Because the collected holiday postcards were in paper for- mat, we first scanned the front and back of each card. We then considered using an optical character recognition (OCR) system to …

Content truncated to 500 words