Identifying Content Types of Messages Related to Open Source Software Projects

Identifying Content Types of Messages Related to Open Source Software Projects Ioannis Korkontzelos, Paul Thompson and Sophia Ananiadou National Centre for Text Mining, School of Computer Science, University of Manchester, UK {ioannis.korkontzelos, paul.thompson, sophia.ananiadou}@manchester.ac.uk Abstract Assessing the suitability of an Open Source Software project for adoption requires not only an analysis of aspects related to the code, such as code quality, frequency of updates and new version releases, but also an evaluation of the quality of support offered in related online forums and issue trackers. Understanding the content types of forum messages and issue trackers can provide information about the extent to which requests are being addressed and issues are being resolved, the percentage of issues that are not being fixed, the cases where the user acknowledged that the issue was successfully resolved, etc. These indicators can provide potential adopters of the OSS with estimates about the level of available support. We present a detailed hierarchy of content types of online forum messages and issue tracker comments and a corpus of messages annotated accordingly. We discuss our experiments to classify forum messages and issue tracker comments into content-related classes, i.e. to assign them to nodes of the hierarchy. The results are very encouraging. Keywords: manually annotated corpus, document classification, open source software 1. Introduction Reusing freely available Open Source Software (OSS) can provide cost effective software solutions to all types of po- tential users, ranging from individuals to large organisa- tions. Although reusing OSS can be beneficial in software development, it is not entirely risk-free, because OSS qual- ity can vary widely (Spinellis and Szyperski, 2004). The quality of an OSS project also depends on maintenance and can vary substantially over time, depending on the interest of the community of its developers and users. Therefore, deciding whether to adopt an OSS project requires an as- sessment of its quality at the time of adoption and possibly an estimate of the perceived enthusiasm of the community towards it in the future. Assessing OSS software quality has traditionally focused on analysing the source code behind the software to cal- culate quality indicators and metrics. However, compli- mentary information about OSS quality can be extracted by analysing messages posted to communication channels (newsgroups, forums, mailing lists), and issue trackers sup- porting OSS projects. For example, analysing online com- munication related to an OSS project can provide informa- tion about the speed at which the community responds to user requests, the rate that bugs are fixed or the satisfaction of users about the responses they receive to their requests. In our previous work, we investigated the task of identi- fying request and responses among online messages about OSS (Korkontzelos and Ananiadou, 2014). This classifica- tion allowed us to compute a preliminary level of metrics that indicate the quality of support offered online. For ex- ample, we computed metrics such as the number of unan- swered threads or issues in a communication channel or the average time taken to respond to a request. In this …

Content truncated to 500 words