Automatically Selected Skip Edges in Conditional Random Fields for Named Entity Recognition

Proceedings of Recent Advances in Natural Language Processing, pages 580–585, Hissar, Bulgaria, 12-14 September 2011. Automatically Selected Skip Edges in Conditional Random Fields for Named Entity Recognition Roman Klinger Department of Bioinformatics Fraunhofer Institute for Algorithms and Scientific Computing Schloss Birlinghoven 53754 Sankt Augustin, Germany roman.klinger@scai.fraunhofer.de Abstract Incorporating distant information via manu- ally selected skip chain templates has been shown to be beneficial for the performance of conditional random field models in con- trast to a simple linear chain based structure (Sutton and McCallum, 2007; Galley, 2006; Liu et al., 2010). The set of properties to be captured by a template is typically man- ually chosen with respect to the application domain. In this paper, a search strategy to find mean- ingful skip chains independent from the application domain is proposed. From a huge set of potentially beneficial templates, some can be shown to have a positive im- pact on the performance. The search for a meaningful graphical structure demon- strates the usefulness of the approach with an increase of nearly 2 % F1 measure on a publicly available data set (Klinger et al., 2008). 1 Introduction Many applications in the field of text segmenta- tion, especially named entity recognition, have been addressed with linear chain conditional ran- dom fields. Using a linear chain of variables to represent the labeling of text is straight forward, as processing text in a sequential manner suggests itself due to the way it is written and firstly per- ceived. While language suggests this linear structure to represent written text, it does not necessarily model all dependencies: Co-referencing a prior entity is an example (while it could be seen as higher order linearity typically pointing back but not forward). Especially in non-scientific texts, in- formation may as well be left out and filled in later to keep a story exciting. Another example is the 10-6 10-5 10-4 10-3 10-2 10-1 0 10 20 30 40 50 60 70 80 Terms with length in % Length of term BioCreative 2 IUPAC CoNLL Figure 1: Distribution of the length of three entity classes. use of filler words as a trivial case where the mean- ing of words can be determined by distant tokens. In named entity recognition, typically linear chain structures of conditional random fields are used. The capabilities of a linear chain such struc- ture may be limited in at least two cases: Firstly, relations between distant tokens can have an im- pact on their meaning. This is a motivation which lead to the previous work presented in the follow- ing Section 2. Secondly, long entity classes can- not be captured as a whole, which is especially interesting because a characteristic of named enti- ties in biology and chemistry is their high length with inter-dependencies between tokens of an en- tity. The distribution of the length of terms in the classes of gene names (BioCreative 2, Smith et al. (2008)), IUPAC names (Klinger et al., 2008) and person names, organizations and places (CoNLL 2005, Sang and …

Content truncated to 500 words