Pivot-Based Topic Models for Low-Resource Lexicon Extraction

Pivot-Based Topic Models for Low-Resource Lexicon Extraction John Richardson† Toshiaki Nakazawa‡ Sadao Kurohashi† †Graduate School of Informatics, Kyoto University, Kyoto 606-8501 ‡Japan Science and Technology Agency, Kawaguchi-shi, Saitama 332-0012 john@nlp.ist.i.kyoto-u.ac.jp, nakazawa@pa.jst.jp, kuro@i.kyoto-u.ac.jp Abstract This paper proposes a range of solutions to the challenges of extracting large and high- quality bilingual lexicons for low-resource language pairs. In such scenarios there is often no parallel or even comparable data available. We design three effective pivot- based approaches inspired by the state-of- the-art technique of bilingual topic mod- elling, extending previous work to take advantage of trilingual data. The pro- posed models are shown to outperform tra- ditional methods significantly and can be adapted based upon the nature of available training data. We demonstrate the accu- racy of these pivot-based approaches in a realistic scenario generating an Icelandic- Korean lexicon from Wikipedia. 1 Introduction Data-driven approaches to natural language processing have been shown to be greatly effec- tive, and the case of bilingual lexicon extraction is no exception. Recent advances in this area have enabled the construction of large, high- quality bilingual lexicons, requiring less parallel data by making use of comparable corpora. While such comparable corpora are readily available for many language pairs, particularly when one of those languages is English, previous direct approaches fail when there is no such data available. For many language pairs there sim- ply does not exist comparable (and even less so parallel) data. Even for languages with a large volume of available parallel data, most corpora cover only limited domains. There are two natural methods to deal with this problem: constructing or mining new data for the direct approach, and finding new ways to make better use of what data is already avail- able. For an example of the construction of comparable corpora, see Zhu et al. (2013). We take the second approach and design pivot-based models for bilingual lexicon extraction. The ma- jor advantage of using a pivot language is that it is possible to take advantage of the large volume of comparable data sharing a common language such as English. In this paper we develop pivot-based ap- proaches to make use of modern bilingual lex- icon extraction methods that can be trained on comparable corpora. We present a selection of efficient algorithms using the framework of topic modelling (Blei et al., 2003). Topic mod- elling has been a popular approach for bilingual lexicon extraction, however its use as a pivot model has yet to be explored. The use of topic models as a semantic similarity measure is a scalable method for low-resource languages be- cause document-aligned comparable pivot train- ing data (such as for English and a low-resource language) is growing ever more widely available. Examples of such sources are Wikipedia, multi- lingual newspaper articles and mined Web data. While there have been many studies on bilin- gual lexicon extraction, there has been little fo- cus on the important problem of resource con- struction for low-resource language pairs. We PACLIC 29 369 29th Pacific …

Content truncated to 500 words