Unsupervised Declarative Knowledge Induction for Constraint-Based Learning of Information Structure in Scientific Documents

Unsupervised Declarative Knowledge Induction for Constraint-Based Learning of Information Structure in Scientific Documents Yufan Guo DTAL University of Cambridge, UK yg244@cam.ac.uk Roi Reichart Technion - IIT Haifa, Israel roiri@ie.technion.ac.il Anna Korhonen DTAL University of Cambridge, UK alk23@cam.ac.uk Abstract Inferring the information structure of scien- tific documents is useful for many NLP appli- cations. Existing approaches to this task re- quire substantial human effort. We propose a framework for constraint learning that re- duces human involvement considerably. Our model uses topic models to identify latent top- ics and their key linguistic features in input documents, induces constraints from this in- formation and maps sentences to their domi- nant information structure categories through a constrained unsupervised model. When the induced constraints are combined with a fully unsupervised model, the resulting model challenges existing lightly supervised feature- based models as well as unsupervised mod- els that use manually constructed declarative knowledge. Our results demonstrate that use- ful declarative knowledge can be learned from data with very limited human involvement. 1 Introduction Automatic analysis of scientific text can help scien- tists find information from literature faster, saving valuable research time. In this paper we focus on the analysis of the information structure (IS) of sci- entific articles where the aim is to assign each unit of an article (typically a sentence) into a category that represents the information type it conveys. By infor- mation structure we refer to a particular type of dis- course structure that focuses on the functional role of a unit in the discourse (Webber et al., 2011). For instance, in the scientific literature, the functional role of a sentence could be the background or moti- vation of the research, the methods used, the experi- ments carried out, the observations on the results, or the author’s conclusions. Readers of scientific literature find information in IS-annotated articles much faster than in unanno- tated articles (Guo et al., 2011b). Argumentative Zoning (AZ) – an information structure scheme that has been applied successfully to many scientific do- mains (Teufel et al., 2009) – has improved tasks such as summarization and information extraction and retrieval (Teufel and Moens, 2002; Tbahriti et al., 2006; Ruch et al., 2007; Liakata et al., 2012; Contractor et al., 2012). Existing approaches to information structure anal- ysis require substantial human effort. Most use feature-based machine learning, such as SVMs and CRFs (e.g. (Teufel and Moens, 2002; Lin et al., 2006; Hirohata et al., 2008; Shatkay et al., 2008; Guo et al., 2010; Liakata et al., 2012)) which rely on thousands of manually annotated training sen- tences. Also the performance of such methods is rather limited: Liakata et al. (2012) reported per- class F-scores ranging from .53 to .76 in the bio- chemistry and chemistry domains and Guo et al. (2013a) reported substantially lower numbers for the challenging Introduction and Discussion sections in biomedical domain. Guo et al. (2013a) recently applied the General- ized Expectation (GE) criterion (Mann and McCal- lum, 2007) to information structure analysis using expert knowledge in the …

Content truncated to 500 words