Data Anonymization for Requirements Quality Analysis: a Reproducible Automatic Error Detection Task

Data Anonymization for Requirements Quality Analysis: a Reproducible Automatic Error Detection Task Juyeon Kang†∗ Jungyeul Park‡∗ † PROMETIL, 52 rue Jacques Babinet. 31100 Toulouse, France. j.kang@prometil.com ‡ CONJECTO, 74 rue de Paris. 35000 Rennes, France. jungyeul@conjecto.com Abstract In this work, we aim at identifying potential problems of ambiguity, completeness, conformity, singularity and readability in system and software requirements specifications. Those problems arise particularly when they are written in a natural language. While we describe them from a linguistic point of view, the business impacts of each potential error are also considered in system engineering context. We investigate and explore error patterns for requirements quality analysis by manually analyzing the corpus. This analysis is based on the requirements grammar that we developed in our previous work. In addition, this paper extends our previous work in a two-fold way: (1) we increase more than twice the number of evaluation data (1K sentences) through a manual verification process, and (2) we anonymize all sensible and confidential entities in evaluation data to make our data publicly available. We also provide the baseline system using conditional random fields for requirements quality analysis, and we obtain 79.47% for the F1 score on proposed evaluation data. Keywords: Requirements quality analysis (ReQA), error detection in ReQA, data anonymization, reproducible task 1. Introduction Among technical documents, requirements are a central is- sue since they must comply with a high number of con- straints of e.g. readability, lack of ambiguity and implicit data, feasibility, relevance, traceability, conformity and overall cohesion and coherence (Firesmith, 2003; Alred et al., 2011). For example, Ambient pressure shall be perma- nently maintained relies too much on the operator’s knowl- edge and practice: what pressure should be maintained to be ambient and what to do in case of interruption? A wrong interpretation may lead to accidents and damages. There exists different types of references for guiding the writing of high quality requirements. Among them, we can cite: • the standards of IEEE (ISO/IEC/IEEE29148:2011), • ARP4754A (Aerospace Recommended Practice), • the recommendations of INCOSE (Guide for Writing Requirements), and • IREB (International Requirements Engineering Board) We can also refer to the principles of controlled natural languages, mainly, those defined in ASD-STE100 (Sim- plified Technical English by Aerospace and Defense In- dustries Association of Europe). Those documents show that requirements must be non-ambiguous consistent, cor- rect, complete, verifiable, singular, readable, feasible and traceable. However, when writing or revising a set of re- quirements, it is particularly challenging to make sure that texts read easily and are unambiguous for any domain ac- tor (Weiss, 1990; Grady, 2013). There are several factors like overload, time missing, novices and specific domain knowledge needed. Tools controlling the authoring quality ∗ Both authors contributed equally to this work. of requirements can be useful for automatically proofread- ing large quantities of requirements. Since we presented the very first linguistic model for requirements quality analysis (ReQA) (Kang and Park, 2016), there are many requests for releasing data. While we build training data from the pub- …

Content truncated to 500 words