Annotation and Analysis of Extractive Summaries for the Kyutech Corpus

Annotation and Analysis of Extractive Summaries for the Kyutech Corpus Takashi Yamamura and Kazutaka Shimada Department of Artificial Inteligence, Kyushu Institute of Technology 680-4 Kawazu Iizuka Fukuoka Japan {t yamamura, shimada}@pluto.ai.kyutech.ac.jp Abstract Summarization of multi-party conversation is one of the important tasks in natural language processing. For conversation summarization tasks, corpora have an important role to analyze characteristics of conversations and to construct a method for summary generation. We are developing a freely available Japanese conversation corpus for a decision-making task. We call it the Kyutech corpus. The current version of the Kyutech corpus contains topic tags of each utterance and reference summaries of each conversation. In this paper, we explain an annotation task of extractive summaries. In the annotation task, we annotate an importance tag for each utterance and link utterances with sentences in reference summaries that already exist in the Kyutech corpus. By using the annotated extractive summaries, we can evaluate extractive summarization methods on the Kyutech corpus. In the experiment, we compare some methods based on machine learning techniques with some features. Keywords: multi-party conversation, the Kyutech corpus, summarization, annotation 1. Introduction Conversation summarization is useful to understand the content of conversations for both participants and non- participants. Many researchers have studied meeting and conversation summarization (Banerjee et al., 2015, Mehdad et al., 2014, Oya et al., 2014). For the summarization tasks, summarization systems need to recognize significant content from each utterance to cover all important information in the conversation. There- fore, corpora are very important to analyze characteristics of conversations and to construct a method for summary generation. There are some corpora in English, such as the AMI corpus (Carletta, 2007) and the ICSI corpus (Janin et al., 2003). These meeting corpora contain meeting record data with many annotations, such as dialogue acts and sum- maries and researchers have effectively used such annota- tions in the summarization task. We have constructed the Kyutech corpus (Yamamura et al., 2016); it is a Japanese conversation corpus about a decision-making task with four participants. To the best of our knowledge, the Kyutech corpus is the first Japanese corpus annotated for summarization tasks and freely avail- able to anyone1. The current Kyutech corpus consists of nine conversations2 with four scenarios of which discus- sion settings differ from each other; topic tags of each ut- terance, and reference summaries of each conversation. On the other hand, the AMI corpus contains numerous annota- tions, such as extractive summaries and dialogue acts. In this paper, we focus on an annotation task of extrac- tive summaries. The purpose of extractive summarization is to select important utterances automatically. We anno- tate an importance tag to each utterance by linking sen- tences in reference summaries. The annotated utterances are extractive summaries for each conversation. By using the extractive summaries, we can apply extractive summa- rization methods that have already been proposed by sev- 1 http://www.pluto.ai.kyutech.ac.jp/˜shimada/resources.html 2 The discussion time is 20 minutes per each conversation. eral researchers to the Kyutech corpus because there has been a …

Content truncated to 500 words