Recovering Casing and Punctuation using Conditional Random Fields

Recovering Casing and Punctuation using Conditional Random Fields Marco Lui, Li Wang NICTA VRL Department of Computing and Information Systems University of Melbourne mhlui@unimelb.edu.au, li@liwang.info Abstract This paper describes the winning entry to the ALTA Shared Task 2013. The theme of the shared task was recovery of cas- ing and punctuation information from de- graded English text. We tackle the task as a sequential labeling problem, jointly learning the casing and punctuation labels. We implement our sequential classifier us- ing conditional random fields, trained us- ing linguistic features extracted with off- the-shelf tools with simple adaptations to the specific task. We show the improve- ment due to adding each feature we con- sider, as well as the improvment due to uti- lizing additional training data beyond that supplied by the shared task organizers. 1 Introduction The ALTA Shared Task 20131 required partici- pants to recover casing and punctuation informa- tion from degraded English text. The task fo- cused specifically on English text where casing and punctuation were entirely absent (i.e. all characters had been reduced to lowercase, and all non-alphanumeric symbols had been omitted). Participants were required to submit word-level predictions, identifying on a word-by-word basis whether (1) the word in its original form has any characters in uppercase, and (2) whether the word is followed by one of a closed set of punctuation marks. Our approach to the task treats the task as a se- quential labeling problem, a common technique in modern natural language processing (NLP). We implement a word-level sequential classifier us- ing conditional random fields (CRFs), a sequen- tial labeling technique that has been successfully 1http://www.alta.asn.au/events/ sharedtask2013 applied to a variety of NLP problems. We make use of a number of linguistic features extracted using off-the-shelf NLP tools, with some simple adaptations to the specific problem at hand. We also make use of additional training data beyond that supplied by the shared task organizers, and show that this has a large impact on the final re- sult. Overall, our approach was the most effec- tive in the shared task by a reasonable margin, out- performing 50 other participants (22 of which out- performed the organizer-supplied baseline). 2 Task Description Participants were required to implement an auto- mated system that could accept as input a stream of words without any casing or punctuation. On the basis of this stream, participants were asked to infer which words in the stream should (1) have some form of casing, and (2) be followed by punc- tuation. The setting was meant to simulate scenar- ios whereby such information is missing, such as from audio transcriptions, or from user-generated content in social media. For purposes of the shared task, the text supplied to participants was an automatically- converted version of original English documents with “standard” casing and punctuation. These documents constitute the “original” goldstandard, and were not supplied to participants. Partici- pants only received the transformed (i.e. lower- cased and punctuation removed) version. Addi- tionally, for training documents, participants were provided with …

Content truncated to 500 words