JAIST Annotated Corpus of Free Conversation

JAIST Annotated Corpus of Free Conversation Kiyoaki Shirai, Tomotaka Fukuoka Japan Advanced Institute of Science and Technology, Nextremer Co., Ltd 1-1 Asahidai, Nomi, Ishikawa, Japan, 1-30-13 Narimasu, Itabashi, Tokyo, Japan kshirai@jaist.ac.jp, tomotaka.fukuoka@nextremer.com Abstract This paper introduces an annotated corpus of free conversations in Japanese. It is manually annotated with two kinds of linguistic information: dialog act and sympathy. First, each utterance in the free conversation is annotated with its dialog act, which is chosen from a coarse-grained set consisting of nine dialog act labels. Cohen’s kappa of the dialog act annotation between two annotators was 0.636. Second, each utterance is judged whether the speaker expresses his/her sympathy or antipathy toward the other participant or the current topic in the conversation. Cohen’s kappa of sympathy tagging was 0.27, indicating the difficulty of the sympathy identification task. As a result, the corpus consists of 92,031 utterances in 97 dialogs. Our corpus is the first annotated corpus of Japanese free conversations that is publicly available. Keywords: Annotated corpus, Dialog Act, Sympathy, Free Conversation 1. Introduction In recent years, study of an open domain conversa- tion system or free conversation system, which can freely talk with users, has attracted much research in- terest (Libin and Libin, 2004; Higashinaka et al., 2014a; Higashinaka et al., 2014b). Unlike a task oriented dialog system, an open domain conversation system can chat with users about various topics. Such systems can be used as robotic pets or nursing care robots that can enrich our daily life. Obviously, language resources are indispensable for the study of free conversation systems. Especially, corpora of free conversations annotated with some linguistic infor- mation are valuable. However, for the Japanese language, there is no annotated corpus in the domain of free conver- sations that is publicly available. This paper introduces an annotated corpus of free conversa- tions in Japanese, called “JAIST Annotated Corpus of Free Conversations”1. It consists of dialogs of two participants, where they freely talk about various topics. Each utterance in the dialogs is annotated with two kinds of tags. One is a dialog act (or speech act), which is the type of utterance that represents the speaker’s intention. The other is sympathy. In this paper, sympathy means that the speaker shows inter- est in the current topic in the conversation. We will report in detail how to construct the corpus as well its statistics. Fur- thermore, two usage cases of this corpus will be reported: the classification of the dialog acts and the identification of sympathy. The rest of the paper is organized as follows. Section 2 will discuss related work. Section 3 will report the details of our corpus, including annotation guidelines, the size of the corpus, the distribution of the tags, and inter-annotator agreement. Section 4 will describe two case studies using our corpus. Finally, the paper is concluded in Section 5 . 2. Related Work One of the well-known dialog corpora is the Switch- board Dialog Act Corpus (University of Colorado at Boul- 1 “JAIST” is the acronym …

Content truncated to 500 words