Extracting Personal Names from Email: Applying Named Entity Recognition to Informal Text

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 443–450, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Extracting Personal Names from Email: Applying Named Entity Recognition to Informal Text Einat Minkov and Richard C. Wang Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15217 {einat,rcwang}@cs.cmu.edu William W. Cohen Ctr for Automated Learning & Discovery Carnegie Mellon University Pittsburgh, PA 15217 wcohen@cs.cmu.edu Abstract There has been little prior work on Named Entity Recognition for ”informal” docu- ments like email. We present two meth- ods for improving performance of per- son name recognizers for email: email- specific structural features and a recall- enhancing method which exploits name repetition across multiple documents. 1 Introduction Named entity recognition (NER), the identification of entity names in free text, is a well-studied prob- lem. In most previous work, NER has been applied to news articles (e.g., (Bikel et al., 1999; McCal- lum and Li, 2003)), scientific articles (e.g., (Craven and Kumlien, 1999; Bunescu and Mooney, 2004)), or web pages (e.g., (Freitag, 1998)). These genres of text share two important properties: documents are written for a fairly broad audience, and writers take care in preparing documents. Important genres that do not share these properties include instant messag- ing logs, newsgroup postings and email messages. We refer to these genres as “informal” text. Informal text is harder to process automatically. Informal documents do not obey strict grammatical conventions. They contain grammatical and spelling errors. Further, since the audience is more restricted, informal documents often use group- and task- specific abbreviations and are not self-contained. Because of these differences, existing NER methods may require modifications to perform well on infor- mal text. In this paper, we investigate NER for informal text with an experimental study of the problem of recognizing personal names in email—a task that is both useful and non-trivial. An application of in- terest is corpus anonymization. Automatic or semi- automatic email anonymization should allow using large amounts of informal text for research purposes, for example, of medical files. Person-name extrac- tion and other NER tasks are helpful for automatic processing of informal text for a large variety of ap- plications (Culotta et al., 2004; Cohen et al., 2005). We first present four corpora of email text, anno- tated with personal names, each roughly compara- ble in size to the MUC-6 corpus1. We experimen- tally evaluate the performance of conditional ran- dom fields (CRF) (Lafferty et al., 2001), a state- of-the art machine-learning based NER methods on these corpora. We then turn to examine the special attributes of email text (vs. newswire) and suggest venues for improving extraction performance. One important observation is that email messages often include some structured, easy-to-recognize names, such as names within a header, names appearing in automatically-generated phrases, as well as names in signature files or sign-offs. We therefore suggest a set of specialized structural features for email; these features are shown to significantly improve perfor- mance on our corpora. We also present …

Content truncated to 500 words