Arabic Corpora for Credibility Analysis

Arabic Corpora for Credibility Analysis Ayman Al Zaatari, Rim El Ballouli, Shady Elbassuoni, Wassim El-Hajj, Hazem Hajj, Khaled Shaban†, Nizar Habash‡, Emad Yehya American University of Beirut, Lebanon, †Qatar University, Qatar, ‡New York University Abu Dhabi, UAE {abz02, roe04, se58, we07, hh63, ehy00}@aub.edu.lb, †khaled.shaban@qu.edu.qa, ‡nizar.habash@nyu.edu Abstract A significant portion of data generated on blogging and microblogging websites is non-credible as shown in many recent studies. To filter out such non-credible information, machine learning can be deployed to build automatic credibility classifiers. However, as in the case with most supervised machine learning approaches, a sufficiently large and accurate training data must be available. In this paper, we focus on building a public Arabic corpus of blogs and microblogs that can be used for credibility classification. We focus on Arabic due to the recent popularity of blogs and microblogs in the Arab World and due to the lack of any such public corpora in Arabic. We discuss our data acquisition approach and annotation process, provide rigid analysis on the annotated data and finally report some results on the effectiveness of our data for credibility classification. Keywords: Credibility, Blogs, Twitter, Crowdsourcing. 1. Introduction The increasing popularity of social networks and blogging websites has transformed the Web into a dynamic, fast- paced and user-centered platform for sharing information; which is commonly referred to now, as the Social Web 2.0. For instance, there are more than 500 million tweets generated daily on Twitter1. Similarly, there are more than 54.2 million blog posts and 52.3 million comments generated every month on just one popular blogging website2. This immense amount of data generated on Twitter and Blogging sites has become a vital and rich source for opinion mining tasks such as sentiment analysis, pro/con classification and emotion recognition. With such a large scale of generated data, it is inevitable that the credibility of the generated data would highly vary. We adopt the Merriam Webster3 definition of credibility that states: credibility is the quality of being believed or accepted as true, real or honest. In other words, a credible (micro)blog is one which holds enough evidence to be believed or accepted as true, real or honest. The presence of non- credible information can highly influence the accuracy of the tasks performed on such data and hence filtering out non-credible data would be very beneficial. To predict the credibility of (micro)blogs, supervised machine learning can be deployed. First, a corpus of (micro)blogs must be constructed. This corpus must then be annotated for credibility (i.e., each (micro)blog in this corpus must be labeled as either credible or not, using human judges). This annotated corpus will then play the role of a training dataset (or ground truth), which can be used to 1 https://business.twitter.com/en-gb/basics 2 https://wordpress.com/activity/ 3 http://www.merriam-webster.com/ build automatic credibility classifiers that can accurately predict the credibility of a given (micro)blog. In this paper, we describe the process of creating two corpora for credibility analysis and we validate their usefulness. The first corpus consists of 175 Arabic blog posts in …

Content truncated to 500 words