A Japanese Chess Commentary Corpus

A Japanese Chess Commentary Corpus Shinsuke Mori1 John Richardson2 Atsushi Ushiku3 Tetsuro Sasada4 Hirotaka Kameko5 Yoshimasa Tsuruoka6 1,4Academic Center for Computing and Media Studies, Kyoto University 2,3Graduate School of Informatics, Kyoto University 5,6Graduate School of Engineering, The University of Tokyo 1,2,3,4Yoshidahonmachi, Sakyo-ku, Kyoto, Japan 5,6Hongo, Bunkyo-ku, Tokyo, Japan 1forest@i.kyoto-u.ac.jp, 2john@nlp.ist.i.kyoto-u.ac.jp, 3ushiku@ar.media.kyoto-u.ac.jp, 4sasada@ar.media.kyoto-u.ac.jp, 5kameko@logos.t.u-tokyo.ac.jp, 6tsuruoka@logos.t.u-tokyo.ac.jp Abstract In recent years there has been a surge of interest in the natural language prosessing related to the real world, such as symbol grounding, language generation, and nonlinguistic data search by natural language queries. In order to concentrate on language ambiguities, we propose to use a well-defined “real world,” that is game states. We built a corpus consisting of pairs of sentences and a game state. The game we focus on is shogi (Japanese chess). We collected 742,286 commentary sentences in Japanese. They are spontaneously generated contrary to natural language annotations in many image datasets provided by human workers on Amazon Mechanical Turk. We defined domain specific named entities and we segmented 2,508 sentences into words manually and annotated each word with a named entity tag. We describe a detailed definition of named entities and show some statistics of our game commentary corpus. We also show the results of the experiments of word segmentation and named entity recognition. The accuracies are as high as those on general domain texts indicating that we are ready to tackle various new problems related to the real world. Keywords: game commentary, named entity, symbol grounding 1. Introduction In recent years there has been a surge of interest in the gen- eration of natural language annotations to describe digital recordings of the real world. A notable example is sen- tence generation from images (Ushiku et al., 2011; Yang et al., 2011). In such studies the natural language anno- tations are often provided by human workers on Amazon Mechanical Turk, and thus are often somewhat artificial. Since Hashimoto et al. (2014) recorded cooking videos of recipes spontaneously posted to an Internet site (Mori et al., 2014b), there have been many other image/video datasets (Ferraro et al., 2015) published. These attempts at connect- ing language expressions to real world objects such as im- ages are often called symbol grounding (Harnad, 1990), an exciting new area in natural language processing. However, images, videos, and many other forms of me- dia have ambiguities that make symbol grounding difficult. In this task we propose to use a well-defined “real world,” that is game states, to concentrate on language ambiguities. The game we focus on is shogi (Japanese chess) (Leggett, 2009). We collected 742,286 commentary sentences in Japanese and then defined domain-specific named entities (NEs), similar to previous work on bio-medical NEs (Set- tles, 2004; Tateisi et al., 2002) or recipe NEs (Mori et al., 2014b). For example, “central rook” is a shogi strat- egy expression similar to “king’s gambit” in chess. We fi- nally annotated NEs for 2,508 sentences to form our game commentary corpus, which has the following distinguish- ing characteristics: • The commentaries …

Content truncated to 500 words