Replicated Study of Effectiveness Evaluation of Cutting-Edge Software Engineering

Abstract

Although many software engineering studies have been conducted, it is not clear whether they meet the needs of software development practitioners. Some studies evaluated the effectiveness of software engineering research by practitioners, to clarify the research satisfies the needs of the practitioners. We performed replicated study of them, recruiting practitioners who mainly belong to SMEs (small and medium-sized enterprises) to the survey. We asked 16 practitioners to evaluate cutting-edge software engineering studies presented in ICSE 2016. In the survey, we set the viewpoint of the evaluation as the effectiveness for the respondent's own work. As a result, the ratio of positive answers (i.e., the answers were greater than 2 on a 5-point scale) was 33.3%, and the ratio was lower than past studies. The result was not affected by the number of employees in the respondent's company, but would be affected by the viewpoint of the evaluation.