Automatic Analysis of Linguistic Features in Journal Articles of Different Academic Impacts with Feature Engineering Techniques

1 Automatic Analysis of Linguistic Features in Journal Articles of Different Academic Impacts with Feature Engineering Techniques Siyu Lei Xi’an Jiaotong University, Xi’an, China The Hong Kong Polytechnic University, Hong Kong, China siyu525.lei@connect.polyu.hk Ruiying Yang* Xi’an Jiaotong University, Xi’an, China yangryd@xjtu.edu.cn Chu-Ren Huang** The Hong Kong Polytechnic University, Hong Kong, China The Hong Kong Polytechnic University-Peking University Research Centre on Chinese Linguistics, Hong Kong, China churen.huang@polyu.edu.hk Abstract English research articles (RAs) are an essential genre in academia, so the attempts to employ NLP to assist the development of academic writing ability have received considerable attention in the last two decades. However, there has been no study employing feature engineering techniques to investigate the linguistic features of RAs of different academic impacts (i.e., the papers of high/moderate citation times published in the journals of high/moderate impact factors). This study attempts to extract micro-level linguistic features in high- and moderate-impact journal RAs, using feature engineering methods. We extracted 25 highly relevant features from the Corpus of English Journal Articles through feature selection methods. All papers in the corpus deal with COVID-19 medical empirical studies. The selected features were then validated of the classification performance in terms of consistency and accuracy through supervised machine learning methods. Results showed that 24 linguistic features such as the overlapping of content words between adjacent sentences, the use of third-person pronouns, auxiliary verbs, tense, emotional words provide consistent and accurate predictions for journal articles with different academic impacts. Lastly, the random forest model is shown to be the best model to fit the relationship between these 24 features and journal articles with high and moderate impacts. These findings can be used to inform academic writing courses and lay the foundation for developing automatic evaluation systems for L2 graduate students. 1 Introduction Academic writing has been considered one of the most difficult and challenging parts for postgraduates with English as their second language. To improve the English learners’ writing ability, many corpus studies compare L2 learners’ writing with native experts’ writing in terms of specific linguistic features and point out the progress that L2 learners should make (e.g., Lorenz, 1999; Hyland, 2008; Qin, 2014). Native experts’ writing in the above studies usually includes high-impact English journal articles (i.e., those with high citations and published in the journals with high-impact factors) because they can be one of the indicators of high-quality academic products and thus become a good reference for assessing students’ academic writing. However, native experts’ publication has been proofread concerning linguistic use, while students’ academic writings have more or less grammatical problems or spelling errors. Directly comparing such two types of writing is inevitably involved in those issues, but many studies do not mention such issues when introducing student corpora (e.g., Qin, 2014; Lu, 2012; Lu & Deng, 2019), which might cause the neglect of some word analysis. Nevertheless, comparison of publications with different impacts in academic fields can overcome such a problem, after all, both have been reviewed and proofread by experts. Grammatical problems might not …

Content truncated to 500 words