Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging

Proceedings of the 27th International Conference on Computational Linguistics, pages 2482–2492 Santa Fe, New Mexico, USA, August 20-26, 2018. 2482 Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging Andrew Matteson Korea University amatteson@korea.ac.kr Chanhee Lee Korea University chanhee0222@korea.ac.kr Young-Bum Kim Amazon Alexa youngbum@amazon.com Heuiseok Lim∗ Korea University limhseok@korea.ac.kr Abstract Due to the fact that Korean is a highly agglutinative, character-rich language, previous work on Korean morphological analysis typically employs the use of sub-character features known as graphemes or otherwise utilizes comprehensive prior linguistic knowledge (i.e., a dictionary of known morphological transformation forms, or actions). These models have been created with the assumption that character-level, dictionary-less morphological analysis was intractable due to the number of actions required. We present, in this study, a multi-stage action-based model that can perform morphological transformation and part-of-speech tagging using arbitrary units of input and apply it to the case of character-level Korean morphological analysis. Among models that do not employ prior linguistic knowledge, we achieve state-of-the-art word and sentence-level tagging accuracy with the Sejong Korean corpus using our proposed data-driven Bi-LSTM model. Title and Abstract in Korean 한국어 형태소 분석 및 품사 부착에 효과적인 문자 단위 정보 한국어는 대표적인 교착어 중 하나이며, 문자 단위 정보의 중요도가 높다. 이에 따라 기존의 한국어 형태소 분석 연구는 자소 정보와 같은 문자 단위 자질을 사용하거나, 형태소 사전 및 변형 규칙과 같은 다량의 언어학적 지식을 활용하였다. 이러한 접근 방법은 사전을 사용하지 않은 문자 단위 형태소 분석은 변형 규칙의 복잡성으로 인해 불가능에 가깝다는 것을 전제로 하고 있다. 이러한 한계를 극복하고자, 본 연구에서는 어떠한 언어 단위도 입력으로 사용할 수 있으며 다단계 변형을 기반으로 형태소 분석 및 품사 부착을 수행하는 방법을 제안한다. 제안된 방법을 적용하여 구현된 데이터 기반 양방향 LSTM 모델의 성능을 세종 말뭉치를 이용하여 정량적으로 평가한 결과, 언어학적 지식을 활용하지 않은 접근 방법들 중 가장 높은 단어 및 문장 단위 부착 정확도를 보임을 확인하였다. 1 Introduction Korean has traditionally posed a challenge for word segmentation and morphological analysis. In addition to virtually unbounded vocabulary sizes, out-of-vocabulary (OOV) rates for models can be high. Korean is an agglutinative, phonetic language with a SOV (Subject-Object-Verb) syntax and a flexible word order, although certain word orders are considered to be “canonical”. Honorifics, conditionals, imperatives, and other forms are all signified using agglutinative endings which sometimes involve transformation of the stem to which they attach. Some endings can be further combined or fused to other endings in a defined order, and furthermore, morphological transformation rules also apply during this process. Transformation rules are mostly consistent at the grapheme level and can be represented by a handful of spelling rules, but many irregular forms do exist. ∗ Corresponding author This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http: //creativecommons.org/licenses/by/4.0/ 2483 # Morphemes 1 나(na)/VV + 는(neun)/ETM 2 날(nal)/VV + 는(neun)/ETM 3 나(na)/NP + 는(neun)/JX 4 나(na)/NNP + 는(neun)/JX 5 나(na)/VX + 는(neun)/ETM Table 1: Ambiguous parses of Eojeol “na-neun” # Eojeol Morphemes (3) 나는 na-neun 나(na)/NP + 는(neun)/JX 하늘에 ha-neul-e 하늘(ha-neul)/NNG + 에(e)/JKB (2) 나는 na-neun 날(nal)/VV + 는(neun)/ETM …

Content truncated to 500 words