Automatic Identification of Learners’ Language Background Based on Their Writing in Czech

International Joint Conference on Natural Language Processing, pages 1428–1436, Nagoya, Japan, 14-18 October 2013. Automatic Identification of Learners’ Language Background based on their Writing in Czech Katsiaryna Aharodnik1, 2 katiaaharodnik@gmail.com Marco Chang1 changreynam1@mail.montclair.edu Anna Feldman1 anna.feldman@montclair.edu Jirka Hana3 jirka.hana@gmail.com 1Montclair State University, New Jersey, USA 2City University of New York, The Graduate Center, New York, USA 3 Charles University, MFF, Czech Republic Abstract The goal of this study is to investigate whether learners’ written data in highly inflectional Czech can suggest a consistent set of clues for automatic identification of the learners’ L1 background. For our experiments, we use texts written by learners of Czech, which have been automatically and manually annotated for errors. We define two clas- ses of learners: speakers of Indo-European lan- guages and speakers of non-Indo-European lan- guages. We use an SVM classifier to perform the binary classification. We show that non-content based features perform well on highly inflectional data. In particular, features reflecting errors in orthography are the most useful, yielding about 89% precision and the same recall. A detailed dis- cussion of the best performing features is provid- ed. 1 Introduction The role of a learner’s native language (L1) in second language (L2) acquisition has been widely discussed in the theories of Second Lan- guage Acquisition (SLA) (Lado, 1957; Rich- ards, 1971; Corder, 1975). The literature sug- gests that writers’ spelling, grammar and lexicon in second languages are often influenced by pat- terns in their native language. However, the ex- tent of the importance of L1 for acquiring L2 still cannot be determined exactly and remains a controversial topic of SLA research. Recently, the availability of learner corpora (e.g., Granger, 2003) has provided opportunities for verifying SLA hypotheses. The previous literature sug- gests that the best performing features for native language identification are largely the features that rely on the content of the data, such as word n-grams, function words and character n-grams (Kochmar, 2011; Koppel et al., 2005; Tsur et al., 2007). This means that future applicability of these features is limited to corpus specific data. The primary goal of our work is to address this problem. We use only non-content based fea- tures, part-of-speech tags (POS) and error tags. Exploring these features is useful for corpora independent approaches to native language iden- tification. Our secondary goal is to analyze the features that perform best for highly inflectional data. We approach binary classification as the beginning step in the development of a system- atic tool for recognizing a specific L1 from morphologically complex L2 data. We use ma- chine learning techniques to identify features contributing to the classification between Indo- European (IE) and non-Indo-European (NIE) L1 backgrounds of learners of L2 Czech. We em- ploy Support Vector Machines (Joachims, 1999) to perform the classification. The results of the experiments show that the non-content based features, especially error tags, are the strongest indicators of the learners’ language background. 2 Related Work The task of native language identification has branched out from authorship attribution and …

Content truncated to 500 words