Designing a Russian Idiom-Annotated Corpus

Designing a Russian Idiom-Annotated Corpus Katsiaryna Aharodnik, Anna Feldman, Jing Peng CUNY, The Graduate Center, Montclair State University New York, New York, Montclair, New Jersey, United States kaharodnik@gradcenter.cuny, {feldmana,pengj}@montclair.edu Abstract This paper describes the development of an idiom-annotated corpus of Russian. The corpus is compiled from freely available resources online and contains texts of different genres. The idiom extraction, annotation procedure, and a pilot experiment using the new corpus are outlined in the paper. Considering the scarcity of publicly available Russian annotated corpora, the corpus is a much-needed resource that can be utilized for literary and linguistic studies, pedagogy as well as for various Natural Language Processing tasks. Keywords: idioms, annotation, corpus, Russian. 1. Introduction In recent years, there has been a growing interest in exploring the questions of automatic processing of semantic relationships and specifically those that are not trivial to define and disambiguate. Among these questions is the problem of automatic identification of figurative language within a large body of text. Largely, the problem lies in the ambiguous nature of idiomatic expressions and identifying the cues for idiom recognition. Some expressions can be interpreted either literally or idiomatically depending on the context in which they occur. Several approaches have been explored in finding a better solution to this problem (e.g., Fazly et al., 2009; Cook et al., 2007; Katz and Giesbrecht, 2006; Sporleder & Li, 2009; Li & Sporleder, 2010; Pradhan et al., 2017; Peng & Feldman, 2016(a, b); Peng et al., 2015; Peng et al., 2014, among others). Unfortunately, the corpora that could be used for training idiom classifiers are scarce, especially if one turns to languages other than English. In this paper, we describe an idiom-annotated corpus for Russian. This corpus is a valuable language resource which can be used for various Natural Language Processing (NLP) tasks, such as automatic idiom recognition. Also, it can be utilized as a pedagogical tool for teaching the intricacies of the Russian language or as a corpus for linguistic investigations. Our corpus is available for research purposes https://github.com/kaharodnik/Ru_idioms. A pilot experiment using the idiom-annotated corpus is also described in the paper. 2. Motivation Idioms lack a clear observable relation between the linguistic meaning and interpretation. Moreover, expressions can be ambiguous between idiomatic and literal interpretation depending on the context in which they occur (e.g., sales hit the roof vs. the roof of the car). Fazly et al.’s (2009) analysis of 60 idioms from the British National Corpus (BNC) has demonstrated that close to half of such expressions have a clear literal meaning; and of those with a literal meaning, on average around 40% of their usages are literal. Therefore, idioms present great challenges for many NLP applications, such as machine translation. There has been substantial computational research on idioms, with an emphasis on English. Previous approaches to idiom detection can be classified into two groups: 1) type-based extraction, i.e., detecting idioms at the type level; 2) token-based detection, i.e., detecting idioms in context. Type-based extraction relies on the idea that …

Content truncated to 500 words