Typed Entity and Relation Annotation on Computer Science Papers

1. Introduction Surveying technical documents such as research papers and patents is a critical task for research scientists, administrators, and policy makers. The automated analysis of technical documents is expected to assist them by retrieving relevant documents and extracting information of interest. In this context, intelligent content-based search systems that can answer queries such as What tasks have CRFs been used for? and What methods have been used for POS tagging? are in demand. For these queries, traditional keyword-based systems are insufficient, as they can only search for the mention of an entity represented by the keywords. These queries search for entities playing a particular role in some context, e.g., CRF as a method in the query What tasks have CRFs been used for? Answers to these queries can be found in various forms in published papers, for example, as phrase-internal structures (CRF-based POS tagging), as sentence-internal structures (CRFs have been successfully applied to POS tagging), and as inter-sentential discourse structures (In this study we propose a new method for the efficiently training CRFs. The proposed method is evaluated for POS tagging tasks). Thus, as several layers of linguistic structure must be investigated in order to answer such queries, we aim to establish a framework for uniformly representing semantic structure involving the roles of entities described in technical documents, represented across those layers. The roles of entities are determined in the context of an event that they are involved in, so that an event-based annotation framework such as the one used in annotating articles in the biomedical domain (e.g., Kim et al., 2008) would be suitable for the current purpose. However, in- domain event framesets in the computer science/technology domain are yet to be established. In addition, because computers and computational methods can be applied to a wide and ever-widening range of topics, formalizing the frameset for events in the computer science/technology domain is an extremely difficult task. Therefore, instead of precisely defining a frameset for in- 1 https://scholar.google.com domain events, we describe the roles of entities in the form of their mutual relations using a set of general relationships such as method-purpose, system-output, and evaluation- result. In addition, we adopt a classification scheme of entities based on their intrinsic nature by anchoring the entity mentions to ontology-based types. This is a common approach taken in biomedical corpus annotation but has not been done for computer science/technology corpus. In this paper, we describe the annotation scheme and initial annotation results, and investigate the effect of entity typing along with the problems in annotation resulting from it. We also evaluate the corpus in a practical setting in application to topic extraction. 2. Related Work The focus of research on searching research papers has been shifting from the social aspects of papers and their authors, such as citation link analysis and co-authorship analysis implemented in search engines such as Google Scholar 1 , to more content-based analysis such as information extraction (IE) concerning the methodological aspects of research papers and patents for …

Content truncated to 500 words