The UIR Uncertainty Corpus for Chinese: Annotating Chinese Microblog Corpus for Uncertainty Identification from Social Media

The UIR Uncertainty Corpus for Chinese: Annotating Chinese Microblog Corpus for Uncertainty Identification from Social Media Binyang Li1, Jun Xiang1, Le Chen1, Xu Han2, 7, Xiaoyan Yu2, Ruifeng Xu3, Tengjiao Wang4, Kam-fai Wong5, 6 1 School of Information Science and Technology, University of International Relations 2 College of Information Engineering, Capital Normal University 3 Harbin Institute of Technology, Shenzhen Graduate School 4 School of Electronics Engineering and Computer Science, Peking University 5 Systems Engineering and Engineering Management, The Chinese University of Hong Kong 6 MoE Key Lab on High Confidence Software Technologies CUHK Sub-Lab 7 Collaborative Innovation Center of Mathematics and Information Science, Capital Normal University {byli, jxiang, lchen}@uir.edu.cn; {hanxu, xyyu}@cnu.edu.cn; xuruifeng@hit.edu.cn; kfwong@se.cuhk.edu.hk Abstract Uncertainty identification is an important semantic processing task, which is critical to the quality of information in terms of factuality in many NLP techniques and applications, such as question answering, information extraction, and so on. Especially in social media, the factuality becomes a primary concern, because the social media texts are usually written wildly. The lack of open uncertainty corpus for Chinese social media contexts bring limitations for many social media oriented applications. In this work, we present the first open uncertainty corpus of microblogs in Chinese, namely, the UIR Uncertainty Corpus (UUC). At current stage, we annotated 40,168 Chinese microblogs from Sina Microblog. The schema of CoNLL 2010 have been adapted, where the corpus contains annotations at each microblog level for uncertainty and 6 sub-classes with 11,071 microblogs under uncertainty. To adapt to the characteristics of social media, we identify the uncertainty based on the contextual uncertain semantics rather than the traditional cue-phrases, and the sub-class could provide more information for research on handing uncertainty in social media texts. The Kappa value indicated that our annotation results were substantially reliable. Keywords: Chinese Microblog, Uncertainty Annotation, UIR Uncertainty Corpus 1. Introduction “Uncertainty - in its most general sense - can be interpreted as lack of information: the receiver of the information (i.e., the hearer or the reader) cannot be certain about some pieces of information” (Szarvas et al. 2012). Many natural language processing problems are required to be solved based on the veracity of information, including question answering, information extraction, and so on, are extraordinarily need the uncertainty identification technique (Li et al., 2014). Most participants of the CoNLL-2010 Shared Task, aiming at identifying uncertain expressions from the biological papers and Wikipedia articles, utilized linguistics features, such as lexical cues, Part-Of-Speech (POS), to detect the uncertain sentences from the given texts. (Farkas et al., 2010). Nowadays, with the advancement of social media, there are more and more text contents containing a large amount of casual or word-of-mouth expressions. The quality of information in social media in terms of factuality has become a primary concern (Wei et al. 2013). Generally speaking, only the factual information with high credibility has the value for use. However, most expressions in social media are published with the hypothesis and episteme which cannot be decided whether it is true or false at …

Content truncated to 500 words