Subword Variation in Text Message Classification

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 510–518, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Subword Variation in Text Message Classification Robert Munro Department of Linguistics Stanford University Stanford, CA 94305 rmunro@stanford.edu Christopher D. Manning Department of Computer Science Stanford University Stanford, CA 94305 manning@stanford.edu Abstract For millions of people in less resourced re- gions of the world, text messages (SMS) pro- vide the only regular contact with their doc- tor. Classifying messages by medical labels supports rapid responses to emergencies, the early identification of epidemics and everyday administration, but challenges include text- brevity, rich morphology, phonological vari- ation, and limited training data. We present a novel system that addresses these, working with a clinic in rural Malawi and texts in the Chichewa language. We show that model- ing morphological and phonological variation leads to a substantial average gain of F=0.206 and an error reduction of up to 63.8% for spe- cific labels, relative to a baseline system opti- mized over word-sequences. By comparison, there is no significant gain when applying the same system to the English translations of the same texts/labels, emphasizing the need for subword modeling in many languages. Lan- guage independent morphological models per- form as accurately as language specific mod- els, indicating a broad deployment potential. 1 Introduction The whole world is texting, but rarely in English. Africa has seen the greatest recent uptake of cell- phones, with an 8-fold increase over the last 5 years and saturation possible in another 5 (Buys et al., 2009). This is a leapfrog technology – for the ma- jority of new users cellphones are the only form of remote communication, surpassing landlines, (non- mobile) internet access and even grid electricity, with costs making texts the dominant communica- tion method. This has led social development orga- nizations to leverage mobile technologies to support health (Leach-Lemens, 2009), banking (Peevers et al., 2008), access to market information (Jagun et al., 2008), literacy (Isbrandt, 2009) and emergency re- sponse (Munro, 2010). The possibility to automate many of these services through text-classification is huge, as are the potential benefits – those with the least resources have the most to gain. However, the data presents many challenges, as text messages are brief, most languages have rich morphology, spellings may be overly-phonetic, and there is often limited training data. We partnered with a medical clinic in rural Malawi and Front- lineSMS:Medic, whose text message management systems serve a patient population of over 2 million in less developed regions of the world. The system allows remote community health workers (CHWs) to communicate directly with more qualified medi- cal staff at centralized clinics, many for the first time. We present a short-message classification sys- tem that incorporates morphological and phono- logical/orthographic variation, with substantial im- provements over a system optimized on word- sequences alone. The average gain is F=0.206 with an error reduction of up to 63.8% for specific labels. For 6 of the 9 labels …

Content truncated to 500 words