KOTONOHA: A Corpus Concordance System for Skewer-Searching NINJAL Corpora

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7077–7083 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7077 KOTONOHA: A Corpus Concordance System for Skewer-Searching NINJAL Corpora Teruaki Oka†, Yuichi Ishimoto†, Yutaka Yagi‡, Takenori Nakamura†, Masayuki Asahara†, Kikuo Maekawa†, Toshinobu Ogiso†, Hanae Koiso†, Kumiko Sakoda†, Nobuko Kibe† †National Institute for Japanese Language and Linguistics, Japan ‡Picolab Inc., Japan kotonoha@ninjal.ac.jp Abstract The National Institute for Japanese Language and Linguistics, Japan (NINJAL, Japan), has developed several types of corpora. For each corpus NINJAL provided an online search environment, ‘Chunagon’, which is a morphological-information-annotation-based concor- dance system made publicly available in 2011. NINJAL has now provided a skewer-search system ‘Kotonoha’ based on the ‘Chunagon’ systems. This system enables querying of multiple corpora by certain categories, such as register type and period. Keywords: corpus concordance system, skewer search, speech corpus, written corpus, diachronic corpus 1. Introduction The National Institute for Japanese Language and Linguis- tics (NINJAL) has developed several corpora in a variety of registers for linguistic research (hereinafter called NINJAL corpora or NINJAL corpus): (1) BCCWJ: Balanced Corpus of Contemporary Written Japanese (2) NWJC: NINJAL Web Japanese Corpus (3) CSJ: Corpus of Spontaneous Japanese (4) CEJC: Corpus of Everyday Japanese Conversation (5) NUCC: Nagoya University Conversation Corpus (6) CWPC: Gen-Nichi-Ken Corpus of Workplace Con- versation (7) COJADS: Corpus of Japanese Dialects (8) CHJ: Corpus of Historical Japanese (9) I-JAS: International Corpus of Japanese as a Second Language In the corpus design of NINJAL, original data such as speeches, videos, or documents are first textised. Speech or video data are manually transcribed into plain text data. Documents such as newspapers, books, and magazines are also digitised using OCR, and manual modifications are made to correct OCR errors. Second, morphological in- formation is annotated to the plain texts. The morphological information is given for each lexical unit prescribed by NINJAL, called the Short Unit Word (SUW, see Figure 1), which is designed for word frequency counts (Den et al., 2008). Since Japanese is not a word- segmented language (does not use white spaces between words), one and the same phrase often may be segmented in several ways. For example, 国立国語研究所 ‘The Na- tional Institute for Japanese Language and Linguistics’ has the following segmentation possibilities: ＼国立国語研究所＼ ＼国立＼国語研究所＼ SUW-Database SUW: 矢張り id: 10551497762939392 Surface: 矢張り Lemma: 矢張り Form: ヤハリ POS: 副詞 `adverb’ Pronunciation: ヤハリ (yahari) ～ ～ SUW: やはり id: 10551497729384960 Surface: やはり Lemma: 矢張り Form: ヤハリ POS: 副詞 `adverb’ Pronunciation: ヤハリ (yahari) ～ ～ SUW: やっぱり id: 10551489139450368 Surface: やっぱり Lemma: 矢張り Form: ヤッパリ POS: 副詞 `adverb’ Pronunciation: ヤッパリ (yappari) ～ ～ Figure 1: Examples of SUW. This shows the SUW: 矢張り (yahari) ‘as was expected’, its orthographic variant やはり (yahari), and their form variant やっぱり (yappari). SUWs are managed by a single database, and the actual entries have columns with more meta-information, such as accent information, but the presentation is simplified here. Individual-search of the Chunagon Skewer-search of the KOTONOHA BCCWJ CSJ ・・・ Individual-search of …

Content truncated to 500 words