FooTweets: A Bilingual Parallel Corpus of World Cup Tweets

FooTweets: A Bilingual Parallel Corpus of World Cup Tweets Henny Sluyter-Gäthje1, Pintu Lohar1, Haithem Afli1-2 and Andy Way1 1ADAPT Centre School of Computing Dublin City University Dublin, Ireland {FirstName.LastName}@adaptcentre.ie 2 Cork Institute of Technology haithem.afli@cit.ie Abstract The way information spreads through society has changed significantly over the past decade with the advent of online social networking. Twitter, one of the most widely used social networking websites, is known as the real-time, public microblogging network where news breaks first. Most users love it for its iconic 140-character limitation and unfiltered feed that show them news and opinions in the form of tweets. Tweets are usually multilingual in nature and of varying quality. However, machine translation (MT) of twitter data is a challenging task especially due to the following two reasons: (i) tweets are informal in nature (i.e., violates linguistic norms), and (ii) parallel resource for twitter data is scarcely available on the Internet. In this paper, we develop FooTweets, a first parallel corpus of tweets for English–German language pair. We extract 4, 000 English tweets from the FIFA 2014 world cup and manually translate them into German with a special focus on the informal nature of the tweets. In addition to this, we also annotate sentiment scores between 0 and 1 to all the tweets depending upon the degree of sentiment associated with them. This data has recently been used to build sentiment translation engines and an extensive evaluation revealed that such a resource is very useful in machine translation of user generated content. Keywords: tweets, parallel data, sentiment translation 1. Introduction Due to the continuously developing Internet technology, there are countless digital media and social networking sites, all of which have a unique characteristic and purpose. Social media has evolved from being cyber world geek buzz to a massive platform for businesses, entrepreneurs, profes- sionals and organizations that seek greater recognition and identification at a very economical price. However, busi- ness information sharing is not the only aspect of web ser- vices, e.g. people from all over the world with different cultural backgrounds stay connected and communicate via widely used social networking websites such as Twitter, In- stagram, Facebook etc. Twitter is an extremely useful so- cial networking tool for different event, business and news organisation that want to reach out to people (and are ready for a reply). Recently, Twitter has gained massive popular- ity and the number of Twitter users has increased signifi- cantly during the last few years. However, Twitter users are often encouraged to write infor- mal texts due to the 140-character limitation.1 They fol- low many other users who tweet in different languages so, the tweets are multilingual in nature and often need to be translated from a specific (source-) language to the (target-) language of choice. In addition, tweets include spelling er- rors, hashtags, user handles, retweets, short forms etc. As a result, translation of such noisy texts becomes a difficult task. To the best of our knowledge, bilingual parallel cor- pora of tweets …

Content truncated to 500 words