Universal Dependencies for Japanese

Universal Dependencies for Japanese Takaaki Tanaka∗, Yusuke Miyao†, Masayuki Asahara♢, Sumire Uematsu†, Hiroshi Kanayama♠, Shinsuke Mori♣, Yuji Matsumoto‡ ∗NTT Communication Science Labolatories, †National Institute of Informatics, ♢National Institute for Japanese Language and Linguistics, ♠IBM Research - Tokyo, ♣Kyoto University, ‡Nara Institute of Science and Technology tanaka.takaaki@lab.ntt.co.jp, yusuke@nii.ac.jp, masayu-a@ninjal.ac.jp, uematsu@nii.ac.jp, hkana@jp.ibm.com, forest@i.kyoto-uac.jp, matsu@is.naist.jp Abstract We present an attempt to port the international syntactic annotation scheme, Universal Dependencies, to the Japanese language in this paper. Since the Japanese syntactic structure is usually annotated on the basis of unique chunk-based dependencies, we first introduce word-based dependencies by using a word unit called the Short Unit Word, which usually corresponds to an entry in the lexicon UniDic. Porting is done by mapping the part-of-speech tagset in UniDic to the universal part-of-speech tagset, and converting a constituent-based treebank to a typed dependency tree. The conversion is not straightforward, and we discuss the problems that arose in the conversion and the current solutions. A treebank consisting of 10,000 sentences was built by converting the existent resources and currently released to the public. Keywords: typed dependencies, Short Unit Word, multiword expression, UniDic 1. Introduction The Universal Dependencies (UD) project has been de- veloping cross-linguistically consistent treebank annota- tion for various languages in recent years. The goal of the project is to facilitate multilingual parser development, cross-lingual learning, and parsing research from a lan- guage typology perspective (Nivre, 2015). The annotation scheme is based on (universal) Stanford dependencies (de Marneffe and Manning, 2008; de Marneffe et al., 2014) and Google universal part-of-speech (POS) tags (UPOS) (Petrov et al., 2012). In our research, we attempt to port the UD annotation scheme to the Japanese language. The traditional annota- tion schemes for the Japanese language have been uniquely developed and are markedly different from other schemes, such as Penn Treebank-style annotation. Japanese syntac- tic parsing trees are usually represented as unlabeled de- pendency structures between bunsetsu chunks (base phrase units), as found in the Kyoto University Text Corpus (Kuro- hashi and Nagao, 2003) and the outputs of syntactic parsers (Kudo and Matsumoto, 2002; Kawahara and Kuro- hashi, 2006). Therefore, we must devise a method to con- struct word-based dependency structures that match the characteristics of the Japanese language (Uchimoto and Den, 2008; Mori et al., 2014; Tanaka and Nagata, 2015) and are able to derive the syntactic information required to assign relation types to dependencies. We describe the conversion from the Japanese POS tagset to the UPOS tagset, the adaptation of the UD annotation for Japanese syntax, and the attempt to build a UD corpus by converting the existing resources. We also address the remaining issues that may emerge when applying the UD scheme to other languages. 2. Word unit The definition of a word unit is indispensable in UD an- notation, which is not a trivial question for Japanese, since a sentence is not segmented into words or morphemes by white space in its orthography. Thus, we have several word unit standards that can be found in corpus annota- …

Content truncated to 500 words