Grammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 40–48, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Grammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings Masahiro Kaneko, Yuya Sakaizawa and Mamoru Komachi Tokyo Metropolitan University {kaneko-masahiro@ed, sakaizawa-yuya@ed, komachi@}.tmu.ac.jp Abstract In this study, we improve grammatical error detection by learning word embed- dings that consider grammaticality and er- ror patterns. Most existing algorithms for learning word embeddings usually model only the syntactic context of words so that classifiers treat erroneous and correct words as similar inputs. We address the problem of contextual information by con- sidering learner errors. Specifically, we propose two models: one model that em- ploys grammatical error patterns and an- other model that considers grammaticality of the target word. We determine gram- maticality of n-gram sequence from the annotated error tags and extract grammat- ical error patterns for word embeddings from large-scale learner corpora. Exper- imental results show that a bidirectional long-short term memory model initialized by our word embeddings achieved the state-of-the-art accuracy by a large mar- gin in an English grammatical error detec- tion task on the First Certificate in English dataset. 1 Introduction Grammatical error detection that can identify the location of errors is useful for second language learners and teachers. It can be seen as a se- quence labeling task, which is typically solved by a supervised approach. For example, Rei and Yannakoudakis (2016) achieved the state-of-the- art accuracy in English grammatical error detec- tion using a bidirectional long-short term memory Phrase pair W2V C&W EWE GWE E&GWE in summer & on summer 0.84 0.75 0.64 0.58 0.54 in summer & in spring 0.84 0.77 0.90 0.80 0.88 in summer & in English 0.40 0.46 0.36 0.25 0.30 on summer & on spring 0.85 0.71 0.82 0.76 0.80 Table 1: Cosine similarity of phrase pairs for each word embedding method. (Bi-LSTM) neural network. Their approach uses word embeddings learned from a large-scale na- tive corpus to address the data sparseness problem of learner corpora. However, most of the word embeddings, in- cluding the one used by Rei and Yannakoudakis (2016), model only the context of the words from a raw corpus written by native speakers, and do not consider specific grammatical errors of language learners. This leads to the problem wherein the word embeddings of correct and incorrect expres- sions tend to be similar (Table 1, columns W2V and C&W) so that the classifier must decide gram- maticality of a word from contextual information with a similar input vector. To address this problem, we introduce two methods: 1) error-specific word embeddings (EWE), which employ grammatical error pat- terns, that is to say the word pairs that learn- ers tend to easily confuse; 2) grammaticality- specific word embeddings (GWE), which con- sider grammatical correctness of n-grams. In this paper, we use the term grammaticality to re- fer to the correct or incorrect label of the tar- get word given its surrounding context. We also combine these methods, …

Content truncated to 500 words