Multiword Expressions and Named Entities in the Wiki50 Corpus

Proceedings of Recent Advances in Natural Language Processing, pages 289–295, Hissar, Bulgaria, 12-14 September 2011. Multiword Expressions and Named Entities in the Wiki50 Corpus Veronika Vincze1, Istv´an Nagy T.2 and G´abor Berend2 1Hungarian Academy of Sciences, Research Group on Artificial Intelligence vinczev@inf.u-szeged.hu 2Department of Informatics, University of Szeged {nistvan,berendg}@inf.u-szeged.hu Abstract Multiword expressions (MWEs) and named entities (NEs) exhibit unique and idiosyncratic features, thus, they often pose a problem to NLP systems. In order to facilitate their identification we developed the first corpus of Wikipedia articles in which several types of mul- tiword expressions and named entities are manually annotated at the same time. The corpus can be used for training or testing MWE-detectors or NER systems, which we illustrate with experiments and it also makes it possible to investigate the co-occurrences of different types of MWEs and NEs within the same domain. 1 Introduction In natural language processing (NLP), a challeng- ing task is the proper treatment of multiword ex- pressions (MWEs). Multiword expressions are lexical items that can be decomposed into sin- gle words and display lexical, syntactic, semantic, pragmatic and/or statistical idiosyncrasy (Sag et al., 2002; Kim, 2008; Calzolari et al., 2002) thus, they often pose a problem to NLP systems. Named Entity Recognition (NER) is another widely researched topic in NLP. There are sev- eral methods developed for many languages and domains, which are tested on manually annotated databases, e.g. the MUC-6 and MUC-7 and the CoNLL-2002/2003 challenges aimed at identify- ing NEs in newswire texts (Grishman and Sund- heim, 1995; Chinchor, 1998; Tjong Kim Sang, 2002; Tjong Kim Sang and De Meulder, 2003). Multiword named entities can be composed of any words or even characters and their meaning cannot be traced back to their parts. For instance, Ford Fo- cus refers to a car and has nothing to do with the original meaning of ford or focus, thus, it is justi- fiable to treat the whole expression as one unit. In this paper, we present our corpus called Wiki50 which contains 50 Wikipedia articles an- notated for multiword expressions and named en- tities. To the best of our knowledge, this is the first corpus in which MWEs and NEs are anno- tated at the same time. We describe the categories occurring in the database, provide some statisti- cal data on their frequency and finally, we demon- strate how noun compounds and named entities can be automatically detected by applying some dictionary-based and machine learning methods. 2 Related corpora and databases Several corpora and databases of MWEs have been constructed for a number of languages. For instance, Nicholson and Baldwin (2008) describe a corpus and a database of English compound nouns (BNC dataset in Table 1). As for mul- tiword verbs, corpora and databases for English (Cook et al., 2008), German (Krenn, 2008), Esto- nian (Muischnek and Kaalep, 2010) and Hungar- ian (Vincze and Csirik, 2010) have been recently developed. The Prague Dependency Treebank is also annotated for multiword expressions (Bejcek and Stran´ak, 2010). As for named …

Content truncated to 500 words