Shrinking Japanese Morphological Analyzers With Neural Networks and Semi-supervised Learning

Proceedings of NAACL-HLT 2019, pages 2744–2755 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2744 Shrinking Japanese Morphological Analyzers With Neural Networks and Semi-supervised Learning Arseny Tolmachev arseny@kotonoha.ws Daisuke Kawahara Kyoto University, Graduate School of Informatics Yoshida Honmachi, Sakyo-ku, Kyoto 606-8501, Japan dk@i.kyoto-u.ac.jp Sadao Kurohashi kuro@i.kyoto-u.ac.jp Abstract For languages without natural word bound- aries, like Japanese and Chinese, word seg- mentation is a prerequisite for downstream analysis. For Japanese, segmentation is often done jointly with part of speech tagging, and this process is usually referred to as morpho- logical analysis. Morphological analyzers are trained on data hand-annotated with segmenta- tion boundaries and part of speech tags. A seg- mentation dictionary or character n-gram infor- mation is also provided as additional inputs to the model. Incorporating this extra informa- tion makes models large. Modern neural mor- phological analyzers can consume gigabytes of memory. We propose a compact alternative to these cumbersome approaches which do not rely on any externally provided n-gram or word representations. The model uses only unigram character embeddings, encodes them using ei- ther stacked bi-LSTM or a self-attention net- work, and independently infers both segmen- tation and part of speech information. The model is trained in an end-to-end and semi- supervised fashion, on labels produced by a state-of-the-art analyzer. We demonstrate that the proposed technique rivals performance of a previous dictionary-based state-of-the-art ap- proach and can even surpass it when training with the combination of human-annotated and automatically-annotated data. Our model it- self is significantly smaller than the dictionary- based one: it uses less than 15 megabytes of space. 1 Introduction Languages with a continuous script, like Japanese and Chinese, do not have natural word boundaries in most cases. Natural language processing for such languages requires to perform some variation of word segmentation. Although some NLP applications, like neu- ral machine translation, started to use unsuper- あ ... る か な い か Encoder ... ... ... ... ... ... ... Dense Softmax Seg B I E t1 t2 tN Figure 1: Proposed model. We encode character uni- gram embeddings into shared representations for each character. The shared representation is projected into a tag-specific representations from which we indepen- dently infer segmentation and per-character tags. vised segmentation methods (Kudo and Richard- son, 2018), resulting segmentation often has deci- sions which are not natural to humans. Supervised segmentation based on a human-defined standard is essential for applications which are designed for interaction on a word-level granularity, for exam- ple, full-text search. Segmentation is commonly done jointly with part of speech (POS) tagging and usually referred to as Morphological Analysis. Modern Japanese Morphological Analyzers (MA) are very accurate, having a >99 segmen- tation tokenwise F1 score on news domain and a >98.5 F1 on web domain (Tolmachev et al., 2018). They often use segmentation dictionaries which define possible words. Also, their models are gen- erally large and unwieldy, spanning hundreds of megabytes in case of traditional symbolic feature- based approaches. Neural models with word or …

Content truncated to 500 words