Feature Extraction for Native Language Identification Using Language Modeling

Proceedings of Recent Advances in Natural Language Processing, pages 298–306, Hissar, Bulgaria, Sep 7–9 2015. Feature Extraction for Native Language Identification Using Language Modeling Vincent Kr´ıˇz, Martin Holub, Pavel Pecina Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics {kriz, holub, pecina}@ufal.mff.cuni.cz Abstract This paper reports on the task of Native Language Identification (NLI). We devel- oped a machine learning system to identify the native language of authors of English texts written by non-native English speak- ers. Our system is based on the language modeling approach and employs cross- entropy scores as features for supervised learning, which leads to a significantly re- duced feature space. Our method uses the SVM learner and achieves the accuracy of 82.4 % with only 55 features. We com- pare our results with the previous similar work by Tetreault et al. (2012) and ana- lyze more details about the use of language modeling for NLI. We experiment with the TOEFL11 corpus (Blanchard et al., 2013) and provide an exact comparison with re- sults achieved in the First Shared Task in NLI (Tetreault et al., 2013). 1 Introduction We present a system for identifying the native lan- guage (L1) of a writer based solely on a sample of their writing in a second language (L2). In this work we focus on English as the second language. According to the weak Contrastive Analysis Hypothesis (Lado, 1957), speakers and writers of the same L1 can sometimes be identified by sim- ilar L2 errors. These errors may be a result of linguistic interference. Common tendencies of a speaker’s L1 are superimposed onto their L2. Na- tive Language Identification (NLI) is an attempt to exploit these errors in order to identify the L1 of the speaker from texts written in L2. In the present study we approach NLI exclusively as a classifi- cation task where the set of the L1 languages is known a priori. 1.1 Motivation and Possible Applications The NLI task is a quickly growing subfield in NLP. The task is motivated by two types of questions: 1. questions about the native language influence in non-native speakers’ speech or writing, and 2. questions about the accuracy of the NLI clas- sification that is achievable, which also in- cludes the technical details of the classifica- tion systems. Native Language Identification can be used in educational settings. It can provide useful feed- back to language learners about their errors. Smith and Swan (2001) showed that speakers of differ- ent languages make different kinds of errors when learning a foreign language. A system which can detect the L1 of the learner will be able to provide more targetted feedback about the error and con- trast it with common properties of the learner’s L1. The knowledge of the native language can be used as a feature for authorship analysis (Sta- matatos, 2009). The plethora of available elec- tronic texts (e.g., e-mail messages, online forum messages, blogs, source code, etc.) presents the potential of authorship analysis in …

Content truncated to 500 words