Weak Semi-Markov CRFs for Noun Phrase Chunking in Informal Text

Weak Semi-Markov CRFs for NP Chunking in Informal Text Aldrian Obaja Muis and Wei Lu Singapore University of Technology and Design {aldrian muis,luwei}@sutd.edu.sg Abstract This paper introduces a new annotated cor- pus based on an existing informal text corpus: the NUS SMS Corpus (Chen and Kan, 2013). The new corpus includes 76,490 noun phrases from 26,500 SMS messages, annotated by uni- versity students. We then explored several graphical models, including a novel variant of the semi-Markov conditional random fields (semi-CRF) for the task of noun phrase chunk- ing. We demonstrated through empirical eval- uations on the new dataset that the new vari- ant yielded similar accuracy but ran in signif- icantly lower running time compared to the conventional semi-CRF. 1 Introduction Processing user-generated text data is getting more popular recently as a way to gather information, such as collecting facts about certain events (Rit- ter et al., 2015), gathering and identifying user pro- files (Layton et al., 2010; Li et al., 2014; Spitters et al., 2015), or extracting information in open domain (Ritter et al., 2012; Mitchell et al., 2015). Most recent work focus on the texts generated through Twitter, which, due to the design of Twitter, contain a lot of announcement-like messages mostly intended for general public. In contrast, SMS was designed as a way to communicate short personal messages to a known person, and hence SMS mes- sages tend to be more conversational and more in- formal compared to tweets. As conversational texts, SMS data often contains references to named entities such as people and lo- cations relevant to certain events. Recognizing those Hmm Dr teh says the research presentation should still prepare, butshe’s not to sure whether they’d time to present Figure 1: Sample SMS, with NPs underlined references will be useful for further NLP tasks. One way to recognize those named entities is to first cre- ate a list of candidates, which can be further filtered to get the desired named entities. Nadeau (Nadeau and Sekine, 2007) lists several methods that work upon candidates for NER. As all named entities are nouns, recognizing noun phrases (NP) is therefore a task that can be potentially useful for further steps in the NLP pipeline to build upon. Figure 1 shows an example SMS message within which noun phrases are highlighted. As can be seen from this example, recognizing the NP information on such a dataset presents some additional challenges over conven- tional NP recognition tasks. Specifically, the texts are highly informal and noisy, with misspelling er- rors and without grammatical structures. The correct casing and punctuation information is often missing. The lack of spaces between adjacent words makes the detection of NP boundaries more challenging. Furthermore, the lack of available annotated data for such informal datasets prevents researchers from understanding what effective models can be used to resolve the above issues. In this work, we focus on tackling these issues while making the following two main contributions: • We build a new corpus of SMS data that is …

Content truncated to 500 words