3arif: A Corpus of Modern Standard and Egyptian Arabic Tweets Annotated for Epistemic Modality Using Interactive Crowdsourcing

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1521–1532, Dublin, Ireland, August 23-29 2014. 3arif: A Corpus of Modern Standard and Egyptian Arabic Tweets Annotated for Epistemic Modality Using Interactive Crowdsourcing Rania Al-Sabbagh†, Roxana Girju†, Jana Diesner‡ †Department of Linguistics and Beckman Institute ‡School of Library and Information Science University of Illinois at Urbana-Champaign, USA {alsabba1, girju, jdiesner} @illinois.edu Abstract We present 3arif1, a large-scale corpus of Modern Standard and Egyptian Arabic tweets annotated for epistemic modality2. To create 3arif, we design an interactive crowdsourcing annotation procedure that splits up the annotation process into a series of simplified questions, dispenses with the requirement for expert linguistic knowledge and captures nested modality triggers and their attributes semi- automatically. 1 Introduction Epistemic modality, according to Palmer (2001), defines the speaker's subjective knowledge, beliefs and judgments about the world's states of affairs. Epistemic modality is used as a linguistic feature for multiple NLP tasks and applications, including sentiment analysis (Abdul-Mageed and Diab 2011), opinion mining (Benamara et al. 2012) and scientific discourse evaluation (Waard and Maat 2012), among others. To-date, there are no large-scale modality-annotated Arabic corpora compared to English (Baker et al. 2010, 2012; Rubinstein et al. 2013), Chinese (Cui and Chi 2013), Portuguese (Hendrickx et al. 2012) and Japanese (Matsuyoshi et al. 2010). The creation of modality-annotated corpora is non-trivial because there is no consensus definition of modality and its attributes in theoretical linguistics to be rendered into annotation tasks and guidelines. Furthermore, most current modality annotation schemes rely on sophisticated theoretically-grounded guidelines that require annotators from linguistics back- ground; hence, annotation is usually restricted to small-scale in-lab settings. In this paper, we present 3arif, a large-scale Arabic corpus annotated for epistemic modality. 3arif comprises 9822 unique tweets in Modern Standard Arabic (MSA) and Egyptian Arabic (EA), annotat- ed for 9966 tokens that map to 214 unique types of epistemic modality. Each epistemic modality is annotated for sense, polarity, intensification, tense, holder(s) and scope(s). The reason that 3arif fea- tures the tweets' genre with an emphasis on MSA and EA tweets is that it comes as part of a larger project to incorporate linguistic features, such as modality, with network-based features to automati- cally identify the key players of Twitter's political discourse in counties of political unrest such as Egypt. We harvested 3arif from a variety of Twitter users including newspapers, TV stations, political campaigns, among others, as well as individuals. As a result 3arif is diglossic for MSA, the formal Arabic variety, and EA, the native Arabic dialect of Egypt. For the annotation of 3arif, we design a simplified procedure that depicts the following ideas: first, it defines each annotation task as a series of open and closed questions that do not require sophisticated linguistics background and, meanwhile, provide annotators with self-explanatory annotation guide- lines; second, it is interactive so that questions are displayed/hidden based on annotators' prior an- swers; and finally, it semi-automatically identifies and merges nested epistemic modality based on an- notators' …

Content truncated to 500 words