An Evaluation of Adopting Language Model as the Checker of Preposition Usage

An Evaluation of Adopting Language Model as the Checker of Preposition Usage Shih-Hung Wu, Chen-Yu Su Dept. of CSIE, Chaoyang University of Technology, Taiwan, R.O.C. shwu@cyut.edu.tw, s9427617@cyut.edu.tw Tian-Jian Jiang, Wen-Lian Hsu Institute of Information Science, Academia Sinica, Taiwan, R.O.C Department of Computer Science, National Tsing-Hua University, Taiwan, R.O.C tmjiang@iis.sinica.edu.tw, hsu@iis.sinica.edu.tw Abstract Many grammar checkers in rule-based approach do not handle errors that come from various usages, for example, the usages of prepositions. To study the behavior of prepositions, we introduce the language model into a grammar- checking task. A language model is trained from a large training corpus, which contains many short phrases. It can be used for detecting and correcting certain types of grammar errors, where local information is sufficient to make decision. We conduct several experiments on finding the correct English prepositions. The experiment results show that the accuracy of open test is 71% and the accuracy of closed test is 89%. The accuracy is 70% on TOEFL-level tests. Keywords: Language Model, Grammar Checker, English preposition usage 1. Introduction Computer-Aided Language Learning is a fascinating area; however, the computer still lacks many abilities of a human teacher, for example, the ability of grammar checking. Technically, it is hard to build a grammar checker that can deal with all types of errors. There are errors caused beyond the knowledge of syntax. For example, to overcome the misusing of prepositions, a system requires more semantic knowledge. There are three major approaches to implement a grammar checker. The first strategy is the syntax-based checking [Jensen et al., 1993]. In this approach, a sentence is parsed into a tree structure. A sentence is correct if it can be parsed completely. Another choice is the statistics- based checking [Attwell, 1987]. In this approach, the system built a list of POS tag sequences based on a POS-annotated corpus. A sentence with known POS tag sequence is considered as a correct one. The last one is the rule-based checking [Naber 2003], where a set of rules is built manually and used to match against a text. Park et al. proposed an online English grammar checker for students who take English as the second language. This system focuses on a limited category of frequently occurring grammatical mistakes in essays written by students. The grammar knowledge is represented in Prolog language. [Park 1997] We find that most grammar checkers do not deal with the errors of preposition usage. We suppose that it should be hard to write rules for all of the prepositions. To evaluate this difficulty, we introduce the language model into the grammar-checking task. Since a language model is usually trained from a large training corpus, it may contain many short phrases with prepositions. The Language Model (LM) is one of the popular natural language processing technology for various applications, like information retrieval, handwriting recognition, speech recognition, and machine translation. [Jurafsky and Martin, 2000] [Manning and Schutze, 1999] An LM uses short history to predict the next word. Word prediction is an essential subtask of …

Content truncated to 500 words