A Generalized LCS Algorithm and Its Application to Corpus Alignment

International Joint Conference on Natural Language Processing, pages 1112–1116, Nagoya, Japan, 14-18 October 2013. A Generalized LCS Algorithm and Its Application to Corpus Alignment Jin-Dong Kim Database Center for Life Science (DBCLS), ROIS, Japan jdkim@dbcls.rois.ac.jp Abstract The paper addresses the problem of text variation which often hinders interopera- ble use or reuse of corpora and annota- tions. A systematic solution is presented based on a variation of Longest Common Sequence algorithm. An empirical exper- iment with 20 full text articles shows it works well with a real world application. 1 Introduction Corpus with annotation is regarded indispensable for the development of natural language process- ing (NLP) technology. As so, many corpora with annotation have been developed, and many groups are working with new annotation projects. As various annotated corpora accumulate in the field, reusability and interoperability is becoming an important issue (Cohen et al., 2005; Johnson et al., 2007; Campos et al., 2012). Among others, Wang et al. (2010) reports that there are a number of corpora that claim to have annotations for pro- tein or gene names, e.g, Genia (Kim et al., 2003), Aimed (Bunescu et al., 2004), and Yapex (Franz´en et al., 2002), and that, however, the protein anno- tations in those corpora are substantially different to each other, which calls for an interoperable in- terpretation of the annotations for integrative reuse of them. Rebholz-Schuhmann et al. (2011) inves- tigates aggregation of variable named entity anno- tations in large scale, which also show the impor- tance of interoperable use of corpus annotation. There also have been efforts for the interop- erability of corpora and annotations from a per- spective of encoding and representation, e.g., Lin- guistic Annotation Framework (LAF) (Ide and Ro- mary, 2004) and Open Linguistics1. Without a doubt, those efforts contribute to improving the in- teroperability of corpora and annotation. 1http://linguistics.okfn.org/ A. T G F - b e t a a c t s · · · 0 1 2 3 4 5 6 7 8 9 10 11 12 (0, 8), Protein B. T G F - & b e t a ; a c t s · · · 0 1 2 3 4 5 6 7 8 9 10 11 12 (0, 10), Signaling molecule C. T G F - β a c t s · · · 0 1 2 3 4 5 6 7 8 9 (0, 5), Protein, Signaling molecule Figure 1: Text variations and annotation to them This paper addresses another type of problem, text variation, which often hinders interoperable use or reuse of corpora and annotations in real world applications. As far as the author knows, it is the first attempt to develop a definite and sys- tematic solution to the problem. The problem of text variant is explained in de- tail in Section 2, while the solutions are presented in Section 3 and 4 After discussions on its real world application in Section 5, the paper is con- cluded in Section 6. 2 Task definition Figure 1 …

Content truncated to 500 words