Stochastic Dependency Parsing of Spontaneous Japanese Spoken Language

Stochastic Dependency Parsing of Spontaneous Japanese Spoken Language Shigeki Matsubara† Takahisa Murase‡ Nobuo Kawaguchi† and Yasuyoshi Inagaki‡ †Information Technology Center/CIAIR, Nagoya University ‡Graduate School of Engineering, Nagoya University Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan matubara@itc.nagoya-u.ac.jp Abstract This paper describes the characteristic features of dependency structures of Japanese spoken language by investigating a spoken dialogue cor- pus, and proposes a stochastic approach to de- pendency parsing. The method can robustly cope with inversion phenomena and bunsetsus which don’t have the head bunsetsu by relax- ing the syntactic dependency constraints. The method acquires in advance the probabilities of dependencies from a spoken dialogue corpus tagged with dependency structures, and pro- vides the most plausible dependency structure for each utterance on the basis of the probabili- ties. An experiment on dependency parsing for driver’s utterances in CIAIR in-car spoken dia- logue corpus has been made. The experimental result has shown our method to be effective for robust parsing of spoken language. 1 Introduction With the recent advances of the continuous speech recognition technology, a considerable number of studies have been made on spoken dialogue systems. For the purpose of smooth interaction with the user, it is necessary for the system to understand the spontaneous speech. Since spoken language includes a lot of gram- matically ill-formed linguistic phenomena such as fillers, hesitations and self-repairs, grammar- oriented approaches are not necessarily suited to spoken language processing. A technique for robust parsing is thus strongly required. This paper describes the characteristic fea- tures of Japanese spoken language on the ba- sis of investigating a large-scale spoken dialogue corpus from the viewpoint of dependency, and moreover, proposes a method of dependency parsing by taking account of such the features. The conventional methods of dependency pars- ing have assumed the following three syntactic constraints (Kurohashi and Nagao, 1994): 1. No dependency is directed from right to left. 2. Dependencies don’t cross each other. 3. Each bunsetsu 1 , except the last one, de- pends on only one bunsetsu. As far as we have investigated the corpus, how- ever, many spoken utterance do not satisfy these constraints because of inversion phenom- ena, bunsetsus which don’t have the head bun- setsu, and so on. Therefore, our parsing method relaxes the first and third ones among the above three constraints, that is, permits the depen- dency direction from right to left and the bun- setsu which doesn’t depend on any bunsetsu. The parsing results are expressed by partial de- pendency structures. The method acquires in advance the proba- bilities of dependencies from a spoken dialogue corpus tagged with dependency structures, and provides the most plausible dependency struc- ture for each utterance on the basis of the prob- abilities. Several techniques for dependency parsing based on stochastic approaches have been proposed so far. Fujio and Matsumoto have used the probability based on the fre- quency of cooccurrence between two bunsetsus for dependency parsing (Fujio and Matsumoto, 1998). Uchimoto et al. have proposed a tech- nique for learning the dependency probability model based on …

Content truncated to 500 words