Overview of the 2013 ALTA Shared Task

Overview of the 2013 ALTA Shared Task Diego Molla Department of Computing Macquarie University Sydney, NSW 2109 diego.molla-aliod@mq.edu.au Abstract The 2013 ALTA shared task was the fourth in the ALTA series of shared tasks, where all participants attempt to solve the same problem using the same data. This year’s shared task was based on the problem of restoring casing and punctuation. As with last year, we used Kaggle in Class as the framework to submit the results and main- taining a leaderboard. There was a strong participation this year, with 50 teams par- ticipating, of which 21 teams submitted re- sults that improved on the published base- line. In this overview we describe the task, the process of building the training set, and the evaluation criteria, and present the results of the submitted systems. We also comment on our experience with us- ing Kaggle in Class. 1 Introduction There are many situations when a piece of En- glish text does not have information about cap- italisation or punctuation. These situations may arise, for example, when the text is the result of automatic transcription from speech, or when the text has been typed in a hurry, such as when tak- ing quick notes or in quick responses in Web fo- rums, or when using media such as text messages. If such a text is to be processed automatically, one may want to restore the missing capitalisation and punctuation, so that the text can be processed using conventional text processing tools and re- sources. It has been shown that introducing a pre- liminary step that automatically restores case in- formation improves the results of machine transla- tion (Lita et al., 2003) and information extraction from speech transcripts (Niu et al., 2004). The task of case and punctuation restoration takes text such as the following as input: ... stored at the ucla television archives the archived episodes were telecast march 8 16 and 24 1971 april 1 and ... The expected output is: ... stored at the UCLA Television Archives. The archived episodes were telecast: March 8, 16, and 24, 1971, April 1 and ... An interesting feature of case and punctuation restoration is that training data can be obtained cheaply. One only needs to take a piece of text and remove case and punctuation. By doing this we can obtain the input data (the text with the case and punctuation information removed) and the tar- get data (the original text). It has been observed that using this approach to generate training data suffices to obtain reasonable results (Niu et al., 2004), and this observation agrees with the results obtained in this shared task, as we will show in this paper. 2 The 2013 ALTA Shared Task Case and punctuation restoration can be formu- lated as a text classification task. Baldwin and Joseph (2009) used a multi-label classification ap- proach where a word can have multiple labels, each label indicating the information to be re- stored in the word. For example, the set …

Content truncated to 500 words