Assessing Relative Sentence Complexity using an Incremental CCG Parser

Proceedings of NAACL-HLT 2016, pages 1051–1057, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Assessing Relative Sentence Complexity using an Incremental CCG Parser Bharat Ram Ambati and Siva Reddy and Mark Steedman School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB bharat.ambati@ed.ac.uk, siva.reddy@ed.ac.uk, steedman@inf.ed.ac.uk Abstract Given a pair of sentences, we present com- putational models to assess if one sentence is simpler to read than the other. While ex- isting models explored the usage of phrase structure features using a non-incremental parser, experimental evidence suggests that the human language processor works incre- mentally. We empirically evaluate if syn- tactic features from an incremental CCG parser are more useful than features from a non-incremental phrase structure parser. Our evaluation on Simple and Standard Wikipedia sentence pairs suggests that incre- mental CCG features are indeed more use- ful than phrase structure features achieving 0.44 points gain in performance. Incremen- tal CCG parser also gives significant improve- ments in speed (12 times faster) in comparison to the phrase structure parser. Furthermore, with the addition of psycholinguistic features, we achieve the strongest result to date re- ported on this task. Our code and data can be downloaded from https://github. com/bharatambati/sent-compl. 1 Introduction The task of assessing text readability aims to clas- sify text into different levels of difficulty, e.g., text comprehensible by a particular age group or second language learners (Petersen and Ostendorf, 2009; Feng, 2010; Vajjala and Meurers, 2014). There have been efforts to automatically simplify Wikipedia to cater its content for children and English language learners (Zhu et al., 2010; Woodsend and Lapata, 2011; Coster and Kauchak, 2011; Wubben et al., 2012; Siddharthan and Mandya, 2014). A related attempt of Vajjala and Meurers (2016) studied the usage of linguistic features for automatic classifi- cation of a pair of sentences – one from Standard Wikipedia and the other its corresponding simplifi- cation from Simple Wikipedia – into COMPLEX and SIMPLE. As syntactic features, they use informa- tion from phrase structure trees produced by a non- incremental parser, and found them useful. However, psycholinguistic theories suggest that humans process text incrementally, i.e., humans build syntactic analysis interactively by enhancing current analysis or choosing an alternative analy- sis on the basis of the plausibility with respect to context (Marslen-Wilson, 1973; Altmann and Steed- man, 1988; Tanenhaus et al., 1995). Besides being cognitively possible, incremental parsing has shown to be useful for many real-time applications such as language modeling for speech recognition (Chelba and Jelinek, 2000; Roark, 2001), modeling text read- ing time (Demberg and Keller, 2008), dialogue sys- tems (Stoness et al., 2004) and machine translation (Schwartz et al., 2011). Furthermore, incremental parsers offer linear time speed. Here we explore the usefulness of incremental parsing for predicting rel- ative sentence readability. Given a pair of sentences – one sentence a sim- plified version of the other – we aim to classify the sentences into SIMPLE or COMPLEX. We use the sentences from Standard Wikipedia (WIKI) paired with their corresponding …

Content truncated to 500 words