Developing an Automated Test of Spoken Japanese

Developing an Automated Test of Spoken Japanese Yasunari Harada Institute for Digital Enhancement of Cognitive Development, Waseda University 1-6-1 Nishi-Waseda, Shinjuku-ku, Tokyo, Japan 169-8050 harada@waseda.jp Masanori Suzuki Ordinate Corporation 1040 Noel Dr. Suite 102, Menlo Park, CA 94025, U.S.A. msuzuki@ordinate.com Jared Bernstein Ordinate Corporation 1040 Noel Dr. Suite 102, Menlo Park, CA 94025, U.S.A. jared@ordinate.com Abstract In order to assess spoken skills of learners of Japanese effectively and more efficiently the Institute for DECODE (Institute for Digital Enhancement of Cognitive Development) at Waseda University is collaborating with Ordinate Corporation to develop and validate an automated test of spoken Japanese, SJT (Spoken Japanese Test). The SJT is intended to measure a test-taker’s facility in spoken Japanese, that is listening and speaking skills in daily conversation, in a quick, accurate and reliable manner. In this paper, we discuss the purposes for developing the SJT, the mechanism of a fully automated test, and the test development processes, including item development and implementation. 1 Introduction According to the Japan’s Agency for Cultural Affairs, in 2002, the number of learners studying Japanese as a second language in Japan was 126,350. This is twice as many students as 10 years ago. Similarly, the Japan Foundation (2003a) reported that a little over 2 million people learned Japanese outside Japan in 2003, which is 18.5 times more than in 1979. The Japan Foundation also administers the Japanese Language Proficiency Test. Approximately 227,000 people took the test in 2001, which is about 4 times more than the number of test-takers in 1996 (The Japan Foundation, 2003b). Currently, one of the major focuses of language instruction is to enhance learners’ ability to communicate, that is, to enhance their oral communication skills. Therefore language assessment should emphasize the competent use of language in spoken communication. Oral Proficiency Interviews (OPIs) are often viewed as assessments well-aligned with this goal. However, administering OPIs is time-consuming and often expensive because each interview may take 20-40 minutes and must be administered and scored by human raters. With the rapid increase of students learning Japanese, there is a growing need for a quick but reliable and accurate assessment instrument in the field of teaching Japanese. However, at present, no such test exists. Ordinate Corporation, a language testing company in California, develops fully automated tests that measure the speaking and listening skills of non-native speakers. The Ordinate testing system is currently delivering tests that measure the spoken language skills of non-native speakers of English and non-native speakers of Spanish. A series of studies has proven that the both tests are highly reliable (The reliability of SET-10 (the Spoken English Test), is 0.97 and the reliability of SST (the Spoken Spanish test) is 0.96). Building on Ordinate’s existing testing system, Ordinate Corporation in the U.S. and the PACLIC 18, December 8th-10th, 2004, Waseda University, Tokyo - 291 - Institute for DECODE at Waseda University in Japan are collaborating to develop a fully automated test of spoken Japanese, the Spoken Japanese Test (SJT). In this paper, we first describe …

Content truncated to 500 words