Multilingual Detection of Personal Employment Status on Twitter

Multilingual Detection of Personal Employment Status on Twitter Manuel Tonneau1,2,3, Dhaval Adjodah1,2,4, João Palotti5, Nir Grinberg6 and Samuel Fraiberger1,2,4 1The World Bank 2New York University 3Centre Marc Bloch 4Massachusetts Institute of Technology 5Qatar Computing Research Institute 6Ben-Gurion University of the Negev Abstract Detecting disclosures of individuals’ employ- ment status on social media can provide valu- able information to match job seekers with suit- able vacancies, offer social protection, or mea- sure labor market flows. However, identify- ing such personal disclosures is a challenging task due to their rarity in a sea of social me- dia content and the variety of linguistic forms used to describe them. Here, we examine three Active Learning (AL) strategies in real-world settings of extreme class imbalance, and iden- tify five types of disclosures about individuals’ employment status (e.g. job loss) in three lan- guages using BERT-based classification mod- els. Our findings show that, even under extreme imbalance settings, a small number of AL itera- tions is sufficient to obtain large and significant gains in precision, recall, and diversity of re- sults compared to a supervised baseline with the same number of labels. We also find that no AL strategy consistently outperforms the rest. Qualitative analysis suggests that AL helps fo- cus the attention mechanism of BERT on core terms and adjust the boundaries of semantic expansion, highlighting the importance of inter- pretable models to provide greater control and visibility into this dynamic learning process. 1 Introduction Up-to-date information on individuals’ employ- ment status is of tremendous value for a wide range of economic decisions, from firms filling job va- cancies to governments designing social protection systems. At the aggregate level, estimates of la- bor market conditions are traditionally based on nationally representative surveys that are costly to produce, especially in low- and middle-income countries (Devarajan, 2013; Jerven, 2013). As so- cial media becomes more ubiquitous all over the world, more individuals can now share their em- ployment status with peers and unlock the social capital of their networks. This, in turn, can provide a new lens to examine the labor market and devise Lost Job Was hired Job ofer Is unemployed Looking for a job “Needing a job cuz I dnt like nt makin money.” Figure 1: An example of a tweet suggestive of its author currently being unemployed and actively looking for a job. policy, especially in countries where traditional measures are lagging or unreliable. A key challenge in using social media to iden- tify personal disclosures of employment status is that such statements are extremely rare in an abun- dance of social media content – roughly one in every 10,000 posts – which renders random sam- pling ineffective and prohibitively costly for the development of a large labeled dataset. On the other hand, simple keyword-based approaches run the risk of providing seemingly high-accuracy clas- sifiers while substantially missing linguistic variety used to describe events such as losing a job, looking for a job, or starting a new position (see Figure 1 for example). …

Content truncated to 500 words