Measuring Innovation in Speech and Language Processing Publications.

Measuring Innovation in Speech and Language Processing Publications. Joseph Mariani1, Gil Francopoulo2, Patrick Paroubek1 1LIMSI, CNRS, Université Paris-Saclay, 2Tagmatica 1rue John von Neumann, 91400 ORSAY (France), 2126 rue de Picpus, 75012 PARIS (France) Joseph.Mariani@limsi.fr, gil.francopoulo@wanadoo.fr, pap@limsi.fr Abstract The goal of this paper is to propose measures of innovation through the study of publications in the field of speech and language processing. It is based on the NLP4NLP corpus, which contains the articles published in major conferences and journals related to speech and language processing over 50 years (1965-2015). It represents 65,003 documents from 34 different sources, conferences and journals, published by 48,894 different authors in 558 events, for a total of more than 270 million words and 324,422 bibliographical references. The data was obtained in textual form or as an image that had to be converted into text. This resulted in a lower quality for the most ancient papers, that we measured through the computation of an unknown word ratio. The multi-word technical terms were automatically extracted after parsing, using a set of general language text corpora. The occurrences, frequencies, existences and presences of the terms were then computed overall, for each year and for each document. It resulted in a list of 3.5 million different terms and 24 million term occurrences. The evolution of the research topics over the year, as reflected by the terms presence, was then computed and we propose a measure of the topic popularity based on this computation. The author(s) who introduced the terms were searched for, together with the year when the term was first introduced and the publication where it was introduced. We then studied the global and evolutional contributions of authors to a given topic. We also studied the global and evolutional contributions of the various publications to a given topic. We finally propose a measure of innovativeness for authors and publications. Keywords: Speech Processing, Natural Language Processing, Text Analytics, Bibliometrics, Scientometrics. 1. Introduction 1.1. Text Analytics of Scientific Papers The application of text analytics to bodies of scientific papers has become an active area of research in recent years (see for example (Ding Y. et al., 2014), (Banchs R.E., 2012)1 or the Saffron2 project. The authors of this paper were invited to conduct several analyses on various conferences: ISCA-Interspeech (J. Mariani et al., 2013), ELRA-LREC (J. Mariani et al., 2014), L&TC (J. Mariani et al., 2015), that they now enlarge to the Speech and Natural Language Processing (SNLP) field in general. They also investigated various aspects of scholar contributions and their evolution over time, such as the production of papers, the collaborations between authors, the citations of papers and authors, the trends in research topics, paper plagiarism and reuse. They proposed measures of the authors’ activity based on paper production, various kinds of centrality in collaboration networks, and paper citation. The present paper makes a link between those analyses and proposes in addition a measure of innovation that could be attached to research topics, authors or publications. It is based on …

Content truncated to 500 words