Word Complexity Estimation for Japanese Lexical Simplification

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3114–3120 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3114 Word Complexity Estimation for Japanese Lexical Simplification Daiki Nishihara†, Tomoyuki Kajiwara‡ † Graduate School of Information Science and Technology, Osaka University, Osaka, Japan ‡ Institute for Datability Science, Osaka University, Osaka, Japan nishihara.daiki@ist.osaka-u.ac.jp, kajiwara@ids.osaka-u.ac.jp Abstract We introduce three language resources for Japanese lexical simplification: 1) a large-scale word complexity lexicon, 2) the first synonym lexicon for converting complex words to simpler ones, and 3) the first toolkit for developing and benchmarking Japanese lexical sim- plification system. Our word complexity lexicon is expanded to a broader vocabulary using a classifier trained on a small, high-quality word complexity lexicon created by Japanese language teachers. Based on this word complexity estimator, we extracted simplified word pairs from a large-scale synonym lexicon and constructed a simplified synonym lexicon useful for lexical simplification. In addition, we developed a Python library that implements automatic evaluation and key methods in each subtask to ease the construction of a lexical simplification pipeline. Experimental results show that the proposed method based on our lexicon achieves the highest performance of Japanese lexical simplification. The current lexical simplification is mainly studied in English, which is rich in language resources such as lexicons and toolkits. The language resources constructed in this study will help advance the lexical simplification system in Japanese. Keywords: Lexical simplification, word complexity, lexicon 1. Introduction Text simplification (Shardlow, 2014) that rewrites a sentence into an easy-to-understand form for lan- guage learners (Petersen and Ostendorf, 2007) and children (Belder and Moens, 2010) is attracting attention. In par- ticular, lexical simplification (Paetzold and Specia, 2017a), which paraphrases complex words into simpler ones ac- cording to the context while maintaining the syntactic struc- ture of the input sentence, is being actively researched (mainly in English). The lexical simplification system is useful not only for assisting the reading comprehension of language learners and children but also for the preprocess- ing of natural language processing applications such as ma- chine translation (ˇStajner and Popovi´c, 2016). Lexical simplification has been studied mainly in En- glish, which is rich in language resources such as a word complexity lexicon (Maddela and Xu, 2018), a paraphrase database from complex phrases to simpler ones (Pavlick and Callison-Burch, 2016) and a toolkit (Paetzold and Specia, 2015). Such language resources are not available for other languages, including Japanese. Lexical simplification consists of the following four sub- tasks (Shardlow, 2014; Paetzold and Specia, 2017a): Complex word identification: Task of deciding which words of a given sentence may not be understood by a given target audience and hence must be simplified. Substitution generation: Task of finding words or expres- sions that could replace the target complex word. Substitution selection: Task of deciding which of the gen- erated candidate substitutions can replace the complex word without compromising the sentence’s grammar or meaning in a given context. Substitution ranking: Task of ranking the remaining can- didate …

Content truncated to 500 words