Extracting the Native Language Signal for Second Language Acquisition

Proceedings of NAACL-HLT 2013, pages 85–94, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Extracting the Native Language Signal for Second Language Acquisition Ben Swanson Brown University Providence, RI chonger@cs.brown.edu Eugene Charniak Brown University Providence, RI ec@cs.brown.edu Abstract We develop a method for effective extraction of linguistic patterns that are differentially ex- pressed based on the native language of the author. This method uses multiple corpora to allow for the removal of data set specific patterns, and addresses both feature relevancy and redundancy. We evaluate different rel- evancy ranking metrics and show that com- mon measures of relevancy can be inappro- priate for data with many rare features. Our feature set is a broad class of syntactic pat- terns, and to better capture the signal we ex- tend the Bayesian Tree Substitution Grammar induction algorithm to a supervised mixture of latent grammars. We show that this extension can be used to extract a larger set of relevant features. 1 Introduction Native Language Identification (NLI) is a classifi- cation task in which a statistical signal is exploited to determine an author’s native language (L1) from their writing in a second language (L2). This aca- demic exercise is often motivated not only by fraud detection or authorship attribution for which L1 can be an informative feature, but also by its potential to assist in Second Language Acquisition (SLA). Our work focuses on the latter application and on the observation that the actual ability to automati- cally determine L1 from text is of limited utility in the SLA domain, where the native language of a stu- dent is either known or easily solicited. Instead, the likely role of NLP in the context of SLA is to pro- vide a set of linguistic patterns that students with certain L1 backgrounds use with a markedly unusual frequency. Experiments have shown that such L1 specific information can be incorporated into lesson plans that improve student performance (Laufer and Girsai, 2008; Horst et al, 2008). This is essentially a feature selection task with the additional caveat that features should be individually discriminative between native languages in order to facilitate the construction of focused educational ex- cersizes. With this goal, we consider metrics for data set dependence, relevancy, and redundancy. We show that measures of relevancy based on mutual in- formation can be inappropriate in problems such as ours where rare features are important. While the majority of the methods that we con- sider generalize to any of the various feature sets employed in NLI, we focus on the use of Tree Sub- stitution Grammar rules as features. Obtaining a compact feature set is possible with the well known Bayesian grammar induction algorithm (Cohn and Blunsom, 2010), but its rich get richer dynamics can make it difficult to find rare features. We extend the induction model to a supervised mixture of latent grammars and show how it can be used to incorpo- rate linguistic knowledge and extract discriminative features more effectively. The end result of this …

Content truncated to 500 words